MirrorSync Advanced Topics

Important:

MirrorSync 6 is no longer eligible for the standard two total hours of free support. It's recommended that before you seek support, you upgrade to the latest version of MirrorSync. If you're seeking support for MirrorSync 6 ($165/hour), email support@360works.com with a clear description of your problem or error.

Welcome to the advanced section of the MirrorSync documentation! This guide will help you customize MirrorSync to suit your development needs, and provide detailed instructions for various situations. If you would like a good general overview of the software, please read through the basic setup documentation first!

Deployment questions

How do I uninstall MirrorSync?

To uninstall on Mac, run the 'Mac Uninstaller.pkg' that comes with the MirrorSync. On Windows, run C:\Program Files\360Works\Uninstall.

On either platform, you can also run the '360Works Admin.jar' application (/Applications/360Works Admin.jar or C:\Program Files\360Works Admin.jar) and remove the MirrorSync instance. However, this does not remove the 360Works service, which is shared by other 360Works applications such as Zulu. You must use the uninstaller to completely remove the 360Works service.

We have plans to implement a Linux uninstaller, but currently the user must manually remove the MirrorSync files and directories, and Tomcat if desired.

Installation for hosting providers

To install multiple instances of MirrorSync, choose the Hosting provider option in the installer. This is exactly the same as the regular installation process, except that it will allow you to rename the instance of MirrorSync. You can continue running the installer as many times as you like, once per client, renaming each instance to something unique. These copies can then be managed via the 360Admin utility, which is found either in your Program Files or Applications folder. When installing additional instances of the application, only a single Tomcat process will be installed which is shared by all the MirrorSync instances. New in MirrorSync 5, log files are now separated by the MirrorSync application name. This is important for hosting providers to be able to see each customers log files separately, and makes troubleshooting with 360Works easier.

Hosting providers should read the next item about using sub-admin accounts with MirrorSync.

If MirrorSync is being installed on a different computer than your FileMaker Server, read the 'split server deployments' section below.

If you are installing MirrorSync on a Linux box, read the section below titled 'How do I install on Linux?' In addition, we recommend customizing your server.xml file and setting the startStopThreads attribute to at least as many instances as you're hosting, to make Tomcat start and stop much faster. Here's an example:

<Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true" startStopThreads="30">

For installation on Mac or Windows, we recommend you use the installer. We at 360Works use the installer for our hosting clients as well, and find it easy to update and manage. If you are curious as to what the installer actually does, it modifies and adds the following:

• Creates a folder at /Library/360Works (Mac) or C:\Program Files\360Works (Windows) that is readable and writeable.
• Downloads and installs an instance of Tomcat 9 into that folder. (Only one copy of Tomcat is installed, regardless of how many copies of MirrorSync are running)
• Copies and renames the installer's MirrorSync.war file, which deploys the web app, as well as the other supporting material into the folder.
• Adds a launch daemon in Library/LaunchDaemons in OS X, or creates a Windows Service to automatically start Tomcat.
• Modifies the http.conf file for Apache to allow for URL redirection. If using Windows with IIS, we create an ISAPI filter or a URL Rewrite rule for the URL redirect.
• Copies a lightweight Admin Utility JAR file to manage Tomcat to either C:\Program Files\360Works or /Applications.

If upgrading from an older version of MirrorSync, we'll remove the URL redirects for the old MirrorSync.

If you prefer to skip the installer and manually deploy using your own instance of Tomcat that you've set up, please note and follow these instructions:

1. Create a folder at /Library/360Works on Mac or C:\Program Files\360Works on Windows. Make sure that it is readable and writeable to the process that Tomcat is running as. This is where MirrorSync stores its private data including the internal sync database and the stored configurations. If you have a strong preference for locating this someplace else, because of disk space for example, configure a '360directory' system property in Tomcat to point to some other path.
2. Rename the MirrorSync.war file to whatever name you'd like the instance to run as for your hosting customer, i.e. MyCustomerSync.war If you decide to run multiple instances of MirrorSync with the same name (using multiple instances of Tomcat), you'll need to set '360directory' separately for each Tomcat instance, otherwise those multiple instances will overwrite each other's private data.
3. Drop that .war file into the webapps directory in your Tomcat instance.
4. Modify the MirrorSync.xml context descriptor and set the administrative username and password for MirrorSync. In Tomcat 6, this file is automatically written to the Tomcat/conf/Catalina/localhost directory. If you are running Tomcat 7, the file is located in the webapps/MirrorSync/META-INF/context.xml file, or you can follow the instructions at this link regarding copyXML to get the same behavior as Tomcat 6 (we recommend doing this, so that you don't lose the context.xml file every time the application is redeployed).
5. If necessary for your configuration, set up URL forwarding from IIS / Apache to your Tomcat connectors. See the Tomcat documentation on how to do this.

Regardless of whether you use the installer or not, if you install enough instances of MirrorSync, you may start to run out of memory. If the memory error messages you get talk about PermGenSpace, then the memory problem is related to loading the MirrorSync classes, not the memory used during actual sync operations. In this case, modify the file at C:\Program Files\360Works\Applications\bin\setenv.bat (Windows) or /Library/360Works/Applications/bin/setenv.sh (Mac) and add this to the list of launch arguments:

-XX:PermSize=64m -XX:MaxPermSize=256m

Silent Installation

Starting in MirrorSync 6.60, we've implemented the ability to silently install MirrorSync. This is useful for those who wish to automate the installation process.

To start, download the installation package to the home directory, unzip it, and run installer.jar using the Java command from the context of this working directory: MirrorSync <VERSION>/Installer data, with your desired arguments. Arguments are how the user or script can control the values that will be input during the silent installation. For Linux installations, you should use the LinuxInstaller.sh script with these same arguments (see this section for example usage of LinuxInstaller.sh.

The only required parameters while running a silent installation are --password <password>, the headlessinstall flag, and the --silent flag itself. Any other parameters that aren't supplied will default to a standard value.

Installation Parameters

Below are the available installation parameters and their effects:

• Silent Installation Mode
• Username
• Password (Required with --silent Flag)
• Reporting Stats
• Hosting Context Name
• Sync Data Location (Linux Only)
• Hostname (Linux Only)

Parameter Details

Silent Installation Mode

• Flag:
  • --silent
• Behavior:
  • Enables silent installation mode, preventing user interaction during installation.
• Usage:
  • This flag does not take a value; it is sufficient to include it in the command.
• Default Value:
  • N/A
• Example:
  • sudo java -jar installer.jar install --headlessinstall --silent --password "MySecurePassword"

---

Username

• Flag:
  • --username <string>
• Behavior:
  • Sets the username for the MirrorSync installation, which will be used to log into the Config Client.
• Usage:
  • Provide a string value for the username.
• Default Value:
  • admin
• Example:
  • sudo java -jar installer.jar install --headlessinstall --silent --username "adminUser" --password "MySecurePassword"

---

Password

• Flag:
  • --password <string>
• Behavior:
  • Sets the password for the installation, which will be used to log into the Config Client. This parameter is required when using the --silent flag.
• Usage:
  • Provide a string value for the password.
• Default Value:
  • N/A - must be provided.
• Example:
  • sudo java -jar installer.jar install --headlessinstall --silent --password "SuperSecret123"

---

Reporting Stats

• Flag:
  • --reportingStats <boolean>
• Behavior:
  • Allows MirrorSync to send diagnostic data.
• Usage:
  • Accepted values: true, yes, 1 (to enable) or false, no, 0 (to disable).
• Default Value:
  • true
• Example:
  • sudo java -jar installer.jar install --headlessinstall --silent --password "MySecurePassword" --reportingStats true

---

Hosting Context Name

• Flag:
  • --hostingContextName <string>
• Behavior:
  • Sets the name of the MirrorSync application. This allows multiple MirrorSync instances to be deployed on the same machine.
• Usage:
  • Provide a non-whitespace string as the application name.
• Default Value:
  • MirrorSync
• Example:
  • sudo java -jar installer.jar install --headlessinstall --silent --password "MySecurePassword" --hostingContextName "CustomMirrorSync"

Note:

You may see outdated documentation specifying that you need to specify a "hostinstall" parameter while using --hostingContextName. This is no longer relevant in current versions of MirrorSync, as of 6.7001+ using the --hostingContextName flag will be enough.

---

Sync Data Location (Linux Only)

• Flag:
  • --syncDataLocation <string>
• Behavior:
  • Specifies the location where sync data will be stored on Linux systems.
• Usage:
  • Provide a valid absolute file path.
• Default Value:
  • /var/lib/360works
• Example:
  • sudo java -jar installer.jar install --headlessinstall --silent --password "MySecurePassword" --syncDataLocation "/opt/MirrorSync/SyncData"

---

Hostname (Linux Only)

• Flag:
  • --serverHostName <string>
• Behavior:
  • Specifies the hostname used by MirrorSync on Linux systems.
• Usage:
  • Provide a string representing the server’s hostname (preferably) or IP address.
• Default Value:
  • The server’s external IP address.
• Example:
  • sudo java -jar installer.jar install --headlessinstall --silent --password "MySecurePassword" --serverHostName "https://sync.example.com"

Example Silent Installer Scripts

Here are some example scripts to help you get started with automating the installation process, organized by platform. All these scripts will install the latest version of MirrorSync. These scripts will also remove the downloaded files and the temporary directory that was created during the installation process.

Note that these examples may need tweaks to work with your specific environment, and you should test your script in a safe environment before implementing it in a production environment.

Sample Windows PowerShell Silent Installation

Here's a sample Windows PowerShell script that will silently install MirrorSync with the username "Lee" and the password "Clementine":

cd ~
# download and unzip MirrorSync
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest "https://secure.360works.com/360Store/WebObjects/360Store.woa/wa/ProductDownload/mirrorsync?v=3" -OutFile "MirrorSync.zip"
mkdir -Force "tmpMirrorSyncDir"
Expand-Archive -Path MirrorSync.zip -DestinationPath tmpMirrorSyncDir -Force
cd "~/tmpMirrorSyncDir"
cd $( Get-ChildItem | Where-Object { $_.Name -like "360Works MirrorSync*" } ).FullName
cd "Installer data"
# install MirrorSync silently with the username Lee and password Clementine
java -jar installer.jar install headlessinstall --silent --username "Lee" --password "Clementine"
# cleanup
cd ~
Remove-Item MirrorSync.zip
Remove-Item -Recurse tmpMirrorSyncDir

Sample Linux Shell Script Silent Installation

Here's a sample Linux shell script that will silently install MirrorSync with the username "Gordon" and the password "Alyx". It's recommended that you put this script into a .sh file that will be run with sudo privileges.

USERNAME="Gordon"
PASSWORD="Alyx"
DOWNLOAD_URL="https://secure.360works.com/360Store/WebObjects/360Store.woa/wa/ProductDownload/mirrorsync?v=3"

cd ~
# install openjdk and tomcat
sudo apt update
sudo apt install openjdk-17-jre-headless tomcat9 tomcat9-admin -y

# download MirrorSync and navigate to the installation package directory
wget -O MirrorSync.zip "$DOWNLOAD_URL"
unzip MirrorSync.zip -d tmpMirrorSyncDir
cd "$(find ./tmpMirrorSyncDir -maxdepth 1 -type d -name "360Works MirrorSync*" -print -quit)"

# make the installer executable
sudo chmod +x LinuxInstaller.sh

# install MirrorSync silently with the specified username and password
sudo ./LinuxInstaller.sh --silent --username "$USERNAME" --password "$PASSWORD"

# cleanup
cd ~
sudo rm MirrorSync.zip
sudo rm -r tmpMirrorSyncDir

This script leverages the LinuxInstaller.sh script that comes with the MirrorSync installation package; LinuxInstaller.sh preps the Linux environment for the installation process. In prior versions its behavior was static, however with the 6.7 update it now accepts the parameters in this list to allow the user or script to configure the installation, or will run as normal (i.e. not silently) if no parameters are passed to it.

Sample macOS Terminal Silent Installation

Here's a sample macOS Terminal script that will silently install MirrorSync with the username "Sonic" and the password "Tails". It's recommended that you put this script into a .sh file that will be run with sudo privileges.

cd ~
# download and unzip MirrorSync
curl -L -o MirrorSync.zip "https://secure.360works.com/360Store/WebObjects/360Store.woa/wa/ProductDownload/mirrorsync?v=3"
unzip MirrorSync.zip -d -o tmpMirrorSyncDir
cd ~/tmpMirrorSyncDir/
cd "$(find . -maxdepth 1 -type d -name "360Works MirrorSync*" -print -quit)/Installer data"
sudo chmod +x installer.jar
# install MirrorSync silently with the username Sonic and password Tails
sudo java -jar installer.jar install headlessinstall --silent --username "Sonic" --password "Tails"
# cleanup
cd ~
rm MirrorSync.zip
rm -r tmpMirrorSyncDir

Using a FileMaker sub-admin account

Unfortunately, the fmsadmin command line tool, which MirrorSync uses for servicing download link requests, does not work with sub-admin accounts. We have reported this as a bug to FileMaker. In the meantime, there is another solution for this problem: In the MirrorSync.xml file, customize the fmsAdminUsername and fmsAdminPassword properties. Insert the full administrative user account information here, and MirrorSync will use that when downloading files or generating download links without prompting the users. In this way, your MirrorSync users can download files without ever needing full FileMaker Server admin credentials.

Split server deployments

If your FileMaker deployment is split onto multiple machines, you can use any of these three configurations:

• Run MirrorSync on just the FileMaker Server, instead of the Web Publishing Engine
• Run MirrorSync on the Web Publishing Engine. If you choose this option, then it is also recommended that you install MirrorSync on the FileMaker Server. This will enable download links to work. You do not need to enter a license key for the extra copy of MirrorSync running on FMS.
• Run MirrorSync on some computer other than the Web Publishing Engine or FileMaker Server. This is a good option if you prefer to deploy MirrorSync on Linux. If you choose this option, then it is also recommended that you install MirrorSync on the FileMaker Server. This will enable download links to work. You do not need to enter a license key for the extra copy of MirrorSync running on FMS.

Regardless of which computer you install on, be sure that when you are configuring MirrorSync, you specify that the Web Publishing and Database Servers are on different computers. This option is on the first screen in the configuration process.

How do I install MirrorSync on Linux?

MirrorSync runs very well in Linux - it's our preferred deployment platform, and it's how we deploy our live demo file.

Starting in MirrorSync 6.5, an installer is packaged with the download of MirrorSync, you can find the instructions here.

Manual instructions:

We assume that you are familiar with Linux, and have already installed Java and Tomcat (version 7 or later should work. We do our testing with Tomcat 9). Here is the setup process:

• Create a directory at /var/lib/360works - make it readable and writeable to Tomcat user (you can put the directory anywhere you want on the filesystem - if you choose a different location, set the '360directory' system property in the MirrorSync.xml file to the location you select).
• Copy the MirrorSync.war file into your webapps directory.
• Wait a few seconds for this to deploy, and then modify the file at ${TOMCAT_HOME}/conf/Catalina/localhost/MirrorSync.xml. Set the adminUsername property, and either the adminPassword or preferably the adminPasswordHash. While you're in this file, skim over the other properties to see if you want to customize any of them. Note: This file may not be called MirrorSync.xml. In some cases the settings file will be in this location and named context.xml instead of MirrorSync.xml. If this is the case, open that file and confirm it is indeed the correct file. If it is you will see the adminUsername and adminPassword properties, amongst other properties . Once you have confirmed that it is the correct file, rename it to MirrorSync.xml and then edit it again, this time adding the values for adminUsername and adminPassword. Tomcat should scan this directory every 10 seconds looking for modifications and if it finds them it will restart the service. This usually takes a few minutes. If you do not see a context.xml file in ${TOMCAT_HOME}/conf/Catalina/localhost/ look in ${TOMCAT_HOME}/webapps/MirrorSync/META-INF for a context.xml file and then copy it from there into ${TOMCAT_HOME}/conf/Catalina/localhost/ , rename it MirrorSync.xml, and then edit it and add values for adminUsername and adminPassword. See the section on MirrorSync.xml for more context.

If desired, configure Apache to forward requests on port 80 to Tomcat, using a ProxyPass directive. Otherwise, just access MirrorSync on the Tomcat port (typically 8080).

Note:

If you are installing MirrorSync on the same Linux machine as FileMaker Server, you will need to add a ProxyPass/Reverse directive because the FileMaker Server installer makes changes to firewalld which blocks port 8080. To do so modify the file at /opt/FileMaker/FileMaker Server/HTTPServer/conf/extra/httpd-proxy.conf and add the following lines to the bottom of the file:

#MirrorSync begin
ProxyPass /MirrorSync http://localhost:8080/MirrorSync timeout=7200
ProxyPassReverse /MirrorSync http://localhost:8080/MirrorSync
#MirrorSync end

Once you have saved these changes stop and start (restart will not reload the config files) the FileMaker Web Server using the fmsadmin utility:

fmsadmin stop httpserver -y
fmsadmin start httpserver -y

Recommended: Either comment out or increase the Timeout setting in the Apache configuration, if it's set to 60 seconds. It is not uncommon for some operations in MirrorSync to take 5-20 minutes, depending on the amount of data involved.

FAIL - Application at context path [/MirrorSync] could not be started

If you see this error in the Tomcat manager when trying to start MirrorSync 6, check the version of Java that Tomcat is using. Tomcat currently installs openJDK 11 which is not currently compatible with MirrorSync 6. You will need to set Tomcat to use JDK 8 by setting the JAVA_HOME variable or using the setenv.sh file.

Does MirrorSync work with runtime versions of FileMaker Pro?

MirrorSync is untested and unsupported for use in this configuration. In addition, the legal licensing agreement for creating runtime versions of FileMaker specifically disallows any automated transfer of data between the runtime version and FileMaker Server. This restriction means that no sync process can legally be used to transfer data between the runtime edition and FileMaker Server, whether that is from a 3rd party or a home-grown automation process. Contact your FileMaker Business Account Manager for volume pricing on FileMaker Pro licenses.

Does MirrorSync work with the M1 Mac?

Yes, MirrorSync is fully supported on the new M1 Macs.

What if I need to redeploy FileMaker Server?

If using a Mac, no additional steps should be required. However, if you are running Windows, you'll want to re-run the installer for MirrorSync after redeploying FileMaker Server.

I am running MirrorSync 4. What do I need to do to upgrade to 5?

MirrorSync 5 is compatible with MirrorSync 4 spoke files - no need to reset sync data or recreate configurations (although some new features, like password saving, will not function until the script is updated).

You can use your existing MirrorSync 5 license key with MirrorSync 6, as long as you have up-to-date maintenance coverage for your MirrorSync 4 maintenance license (12 months from the date of purchase for MirrorSync is automatically covered, and can be renewed each year for 25% of the retail value).

Updating MirrorSync only requires you to double-click the installer and follow the instructions. You do not need to uninstall the previous version. In fact, if you do, you will lose any customizations you have made to the MirrorSync.xml file.

If you encounter any problems during syncing, PLEASE use the 'send problem report' link on the MirrorSync launch page. This is much more useful than emailing us. If you do email us, we will probably ask you to resubmit using that link, because that sends us the detailed sync log file, which is necessary to solve most tech support issues.

Installing without a network connection

The MirrorSync installer normally needs a network connection during the installation process. It uses this to download a copy of Apache Tomcat 9 from the Apache website, as well as doing a network license check to validate the license key. If you are in an environment that has no outbound network capabilities, you can still install MirrorSync, but you'll need to follow these extra steps:

1. Contact support@360works.com and ask for a customized build of MirrorSync that does not need a network connection for license checking. Be sure to include your license key and registration information.
2. Manually create a new folder at /Library/360Works (on Mac) or C:\Program Files\360Works (on Windows).
3. Download Apache Tomcat 9 at this link and extract it into the 360Works directory.

Now you should be able to run the installer without any network connection.

How do I migrate MirrorSync from one server to another?

Start by determining which scenario you fall into:

1. My MirrorSync address is a DNS name (not an IP address which will need to change), and my hub database server is either running on the same computer, or is also a DNS name.
2. My MirrorSync address is an IP address which will need to change, AND I'm running a client-to-server sync.
3. My hub database server is an IP address which will need to change, and I'm running a server-to-server sync.

If you're in scenario #1, you're in luck. The steps are pretty simple:

1. Stop the 360Works Service on the old server by using the utility at C:\Program Files\360Works\ or /Applications/360Work Admin. Click Stop at the top of the window to stop the service
2. Install MirrorSync on the new computer.
3. Once the installation is complete, stop MirrorSync using the 360Works Admin utility by selecting it in the list and clicking stop at the bottom of the window.
4. Copy the sync data folder on the old server at C:\Program Files\360Works\SyncData4_MirrorSync or /Library/360Works/SyncData4_MirrorSync to the same location on the new server, overwrite the existing folder. Make sure you have matched the file permissions from the previous computer.
5. Start MirrorSync back up using the 360Works Admin utility by clicking on MirrorSync and then clicking on start at the bottom of the window
6. Once the new instance is up and running and syncing, you can delete your old installation on the old server

For scenario #2, you're out of luck. You can export and import the configuration, so you don't have to create the configuration. However, none of your spoke users will be able to sync until you edit the MirrorSync configuration, modify your MirrorSync script as outlined in the configuration process, and distribute new spoke copies to your users. Next time use a hostname instead of an IP address.

For scenario #3, the steps are similar to #1, but you'll need to add a folder re-name step:

1. Stop the 360Works Service on the old server by using the utility at C:\Program Files\360Works\ or /Applications/360Work Admin. Click Stop at the top of the window to stop the service
2. Install MirrorSync on the new computer.
3. Once the installation is complete, stop MirrorSync using the 360Works Admin utility by selecting it in the list and clicking stop at the bottom of the window.
4. Copy the sync data folder on the old server at C:\Program Files\360Works\SyncData4_MirrorSync or /Library/360Works/SyncData4_MirrorSync to the same location on the new server, overwrite the existing folder. Make sure you have matched the file permissions from the previous computer.
5. This is the extra step Look in the sync data directory for a subdirectory with the old IP address of your server. Re-name the folder to the new IP address of your server.
6. Start MirrorSync back up using the 360Works Admin utility by clicking on MirrorSync and then clicking on start at the bottom of the window
7. Once the new instance is up and running and syncing, you can delete your old installation on the old server

Replacing your hub database with a spoke copy

If something happens to your hub database, you might want to replace it with a spoke copy (either from a client or a server in a server-to-server sync) that has the same data as the hub. There are two basic approaches: 1) Reset the sync data, or 2) Don't reset the sync data.

For either option, start with these steps:

• Make sure the spoke database is as up-to-date as possible. If possible, run one final sync to bring any recent changes from the hub to the spoke.
• Make sure you have backup copies of both the hub and spoke database.
• Stop any auto-sync that is currently running

Info:

The following instructions reference different types of MirrorSync records. This refers to the value of the MirrorSync::type field; you can view this value in the "Internal" tab of the MirrorSync layout. These instructions will also reference the sync4 field in the MirrorSync table. You can see where to find both of these values in the following screenshot:

MirrorSync Layout Internal Tab

Resetting the sync data: This is the most reliable option. If there are MirrorSync-managed keys in any table, this is the ONLY option. If there are multiple spokes, they should all be replaced after this. Here are the steps to take:

• Delete the 'hub' and 'client' records from the spoke MirrorSync table.
• For the 'server' record, set the MirrorSync::sync4 field to the current timestamp, in the format 2025-03-11 14:02:25
• Move the spoke database to the hub (if it's a server-to-server sync, use scp for Linux or SMB for Windows - see 'Sharing files between Windows instances'). If there are external containers, it should be OK to skip them.
• Reset the sync data
• Run initial sync - there shouldn't be a dialog asking what to do about records in the spoke. If there is, you may still have a 'client' record in the spoke MirrorSync table.
• After that finishes, run a few incremental syncs, then enable auto-sync if appropriate.

Not resetting the sync data: This may be an OK option if it's a big problem to distribute new spoke copies to users. It requires all tables to be developer-managed. One big downside of this approach is that if the record order does not match between the old hub database and the replacement spoke database, all deletion scans will be "slow" instead of "fast" until the sync data is reset.

• Move the spoke database to the hub (if it's a server-to-server sync, use scp for Linux or SMB for Windows - see 'Sharing files between Windows instances'). If there are external containers, it should be OK to skip them.
• On the hub copy only, delete the 'client' record from MirrorSync table
• Run a sync to test that all is working. If so, enable auto-sync if appropriate.

Integrating MirrorSync with External Volumes on macOS

Mac OS X Catalina and later implement more restrictive security measures when it comes to running Java applications. A possible scenario where these security measures may inhibit MirrorSync is if you want to utilize the feature to automatically deploy updated versions of the hub database to the spoke server in a FileMaker Server-to-FileMaker Server sync, and the databases are hosted on a separate volume from the boot drive.

To allow MirrorSync to communicate with external volumes on the Mac server where it is installed, you will need to enable Full Disk Access permissions to Java.

While on the Mac server where MirrorSync is installed, perform the following tasks:

1. Open System Preferences
2. Click on the Security & Privacy module
3. Navigate to the Privacy tab
4. Find and select 'Full Disk Access' from the category menu on the left
5. Click the lock and enter your user password to authorize system changes
6. Check the box next to 'java'

This will let the OS know to allow Java applications, such as MirrorSync, access to external volumes. See the image for a visual representation:

Example Security and Privacy Settings Dialog

Networking questions

Can MirrorSync send encrypted data? What about SSL enabled FMS?

Yes, it can. There are two network channels for encryption: The connection from the syncing client to MirrorSync, and the connection from MirrorSync to FileMaker Server. SSL encryption can be enabled or disabled for each channel separately. In most cases, the connection from the syncing client to MirrorSync is more important for encrypting, since the connection between MirrorSync and FileMaker Server is usually on the same computer or the same LAN.

To encrypt data between the syncing client and MirrorSync, make sure that you have a valid SSL connection installed on the web server where MirrorSync is installed. You can test this by going to https://yourServerAddress/MirrorSync. Once that is working, all you need to do is check the SSL box in the spoke configuration screen in MirrorSync.

To encrypt data between MirrorSync and FileMaker Server, follow the documentation from FileMaker on enabling SSL encryption in FileMaker Server. Once that is complete, JDBC communication is automatically encrypted. If you choose the XML option, be sure to check the SSL checkbox in the MirrorSync hub configuration screen. If you select the XML option, and get an error saying that you have a self-signed certificate, then you will need to follow the Custom SSL Certificate setup instructions. You may also need to follow the instructions in our Configuring IIS for MirrorSync over SSL tutorial to be sure your IIS configuration is set up correctly. MirrorSync configures these during installation but doesn't always succeed.

I need my database to be HIPAA-compliant, what should I do with MirrorSync?

HIPAA compliance depends on a large number of requirements. However, one of those requirements is to transmit only encrypted data. See the question above if you experience difficulties. Please keep in mind that encryption is one piece of HIPAA compliance, so consult the Department of Health and Human Services for more information.

What ports are required for MirrorSync? Can I change them?

During the MirrorSync configuration client only requires port 80, although it will connect more quickly if all ports are open.

For the connection between MirrorSync and your server, it depends on which type of database server you're using:

• FileMaker Server with XML: 80 and 443 (if using SSL)
• FileMaker Server with JDBC: 2399
• MySQL: 3306
• MS SQL: 1433
• Oracle: 1521

For the connection between MirrorSync and spoke devices running FileMaker Pro or Go, MirrorSync transmits all non-container data (text, number, date, time, timestamps) on standard HTTP ports 80 or 443 (if SSL encryption is being used).

Container data is also transmitted over standard HTTP ports, if 1) you are running MirrorSync 2.1 or later, 2) using FileMaker 13 Pro or Go, and 3) the container data is 10 megabytes or smaller. This 10 megabyte limit can be adjusted by setting the $$base64Limit global variable in the MirrorSync script. Contact support@360works.com for more information on this setting.

If these three conditions are met, MirrorSync will only use standard ports 80 or 443, which are allowed by default on almost all firewalls.

If these three conditions are not met, MirrorSync will connect as a guest of FileMaker Server over port 5003, which is the regular port for communication between FileMaker Pro and FileMaker Server. This will likely require custom configuration of your company firewall, if you have one.

Will MirrorSync work on a VPN?

Yes. MirrorSync runs over standard HTTP protocols (which are VPN compatible) for non-container data, and standard FileMaker protocols (which are also VPN compatible) for container data.

Is MirrorSync compatible with Network Address Translation (NAT)?

Yes. When configuring MirrorSync, select the option that says 'My internal and external IP addresses are different.' This will write the sync script so that if it detects it's running on the same LAN as MirrorSync, it will use the internal IP address, and if it's running outside the MirrorSync LAN, it will automatically switch to the external IP address.

Sometimes, it is impossible for the MirrorSync script to tell whether it is on the same LAN as the MirrorSync server. For instance, if MirrorSync is configured on a network where the MirrorSync server is running at IP address 192.168.1.5, and then a user tries to sync from some other network with the same IP address range (192.168.x.x and 10.1.1.x are common IP address ranges), the MirrorSync script will think that it's on the same LAN as the server, and will try the internal IP address, which will fail. If this happens, it will try for 60 seconds and then automatically switch to the external network. If the external network succeeds, this will be remembered for as long as the file is open, so future syncs will try the external network first.

If you know that the user is on an external network (for example, if you present a dialog asking the user whether they are inside the LAN or not), you can force MirrorSync to try external before internal or vice-versa. To do this, set the global variable $$MIRRORSYNC_CONNECTION to the word 'internal' or 'external' before running the MirrorSync script. This will allow you to override the normal search order.

Primary Key / Serial Numbers

What is the difference between 'MirrorSync-managed' and 'Developer-managed' primary keys? Do I need to change how I do my primary keys? Which one should I pick?

Before answering this question, it's first necessary to explain why primary keys are a complex issue with synchronization. With traditional incrementing serial numbers, one database might have records numbered 1 through 10. Another database might have records 1 through 50. If we create a new record on the first database, it will get assigned the next number in the sequence (11). However, if we try to write that record to the second database, it will conflict with record 11 that already exists there. Here are several approaches to solving that problem:

1. Use Universally Unique Identifiers (UUIDs) as primary keys. These are typically long, 36-character strings that look like this: "D2EF9F69-5DEA-4FE3-9095-162C77F76FBF". They are sufficiently random to statistically eliminate the possibility of duplicate values. This makes them ideal for syncing databases - you can safely write records from one database to another without worrying about conflicting IDs. MirrorSync (and most other sync frameworks) supports UUIDs.
2. Combine a traditional serial number with some delimiter, such as each user's initials or a file ID. This way, user 1 would have primary keys "1.1", "1.2", "1.3", and so on. User 2 would create primary keys "2.1", "2.2", "2.3", etc. This has the advantage of being shorter and more readable than UUIDs, but the added management consideration of assigning unique identifiers to each user. MirrorSync can help to configure this for you, as explained later.
3. A variation on the second solution would be to assign each user a particular numeric range, so that user 1 generates primary keys in the range of 1-10,000; user 2 generates primary keys 10,001-20,000; and so on. This has the advantage of pure numeric values (instead of text), but the disadvantage of the possibility of conflicts if a user exceeds the expected number of records. MirrorSync can help with this approach as well.
4. A very different approach is to allow conflicting primary keys to exist on each separate database, without ever writing those primary keys to other databases. When record #11 is written from the first database to the second (in our original example), instead of being written with primary key 11, it is written with primary 51 (the next number in the sequence on the second database). This has the advantage of the shortest possible primary keys, which are pure numeric values, with no possibility of conflicts. It is also the way that the majority of existing databases are designed. MirrorSync supports this method (and is the only sync framework that does, to our knowledge). It creates an internal table to translate between the primary keys on all database that are syncing, so that when record #11 is later updated, MirrorSync knows to change record #51 in the second database. MirrorSync also re-writes foreign keys when they are written from one database to another, so that foreign keys that contained '11' on the first database will be re-written with '51' in the second database.

Options 1, 2, and 3 are considered 'Developer-managed', because MirrorSync writes the primary keys unmodified between the databases being synced. It is the developer's responsibility to pick some scheme that ensures that the same primary key is never used for different records on different databases. There are other variations on the same theme (i.e. one database gets odd numbers and the other gets even numbers), but they all are treated the same by MirrorSync. Developer-managed keys must NOT have the 'prohibit modification' option set, because MirrorSync needs to write to this field (to make it match the device where the record originated).

Option 4 is considered 'MirrorSync-managed.' The developer is not responsible for the uniqueness of primary keys across different databases; MirrorSync takes care of this for you. You may enable the 'prohibit modification' option for MirrorSync-managed keys, because MirrorSync only reads from this field, it does not write to it.

If you have no preference for serial numbers or UUIDs, we generally recommend UUIDs. The advantage to UUIDs is that they can easily be merged manually in the event that the sync does not work for some reason. However, serial numbers are fully supported, and if it's difficult to switch to UUIDs, stick with what you have. If you need 'user-friendly' numbers for things like invoice numbers, job numbers, or check numbers, read the next FAQ on user-friendly serial numbers before making a decision.

We highly recommend AGAINST using any meaningful data in your primary key. This applies to data modeling in general, not just MirrorSync. This means that primary keys should not be based on anybody's name, initials, social security, office location, company, username, etc. However, if you must use a schema like this, you should select 'Developer-managed' primary keys. Remember that when using developer-managed keys, YOU are responsible for making sure that the same primary key is never created in the same table on more than one device.

These are some common mistakes made with primary keys when syncing:

• If your database is currently designed with serial numbers for primary keys, do not try to short-cut the solution by simply creating a new UUID field and try to trick MirrorSync into using that as your primary key. MirrorSync won't re-number your foreign keys, which means that the wrong children records will point to the wrong parent records when the records are synced with the server.
• If you use a calculated value for your primary key, for instance by combining other fields or be adding a prefix, select 'Developer Managed' primary keys in the MirrorSync configuration. You are then responsible for making sure that the primary keys are unique across all devices. Only use 'MirrorSync Managed' primary keys with a single text or number field, such as a serial number.

What if I need to assign a user-friendly serial number to my records that stays the same when it is synced? For example, an invoice number?

The problem with MirrorSync-managed serial numbers is that they are not suitable for user-visible numbers, such as invoice numbers. That's because the primary key will be different on one device than another (read section above for explanation). If you're using the primary key serial number as your invoice number, it is clearly a problem if your invoice number is different between your laptop and the server!

There are several solutions to this problem. One of the first things to establish is whether your database uses a single field as both the primary key and the user-visible value (we'll refer to it as the 'invoice number'). It is always preferable (even when not syncing) to have these be separate fields from each other. Invoice numbers / job numbers / user visible numbers should NOT be what you use as a primary key in your database. Primary keys are internal database identifiers, and should not do double-duty as a user-readable value.

If the fields are separate, the problem becomes simpler to solve, because you have the flexibility to change the value of the invoice number without breaking relationships. See the section below on write-back values for the recommended approach.

If the same field is being used for the primary key and the invoice number, and if it is feasible to split this into two separate fields (one for relationships, and one for display/searching), you should do so. Unfortunately, this can sometimes be a very big job - creating the new field is easy, but then you either need to find every place that that field is used in the user interface and point it to the newly created invoice number field, or else you need to find every place that it is used in the relationship graph and re-point that to the newly created primary key field. Once you've split the single primary key field into two fields (a primary key and an invoice number field), follow the instructions below for configuring the invoice number field as a write-back value. If you'd like, 360Works can quote a price to do this development work for you.

We recommend splitting the fields as describe above. However, if this is just not feasible, there is a workaround - see the section below on MIRRORSYNC_DATABASEID for the workaround.

Write-back values for user-visible numbers

Let's say that you have a UUID as your primary key field, as well as an invoice number field. Using the write-back approach, the invoice number field will be blank when a record is created in a spoke file. When that invoice record is synced to the server, it will get assigned the next invoice number, and that number will be written back to the invoice number field on the spoke file. This approach does not work unless the invoice number field is separate from the primary key.

Advantages of this approach:

• Invoice numbers are in a simple, short numeric sequence, just like they would be without syncing.

Disadvantages:

• The spoke file does not get an invoice number until the record is synced for the first time. If this does not work in your use case, see 'MIRRORSYNC_DATABASEID' below for an alternative approach.

To use this approach, follow these steps:

• If you do not already have a serial number field in your table, create one. If you have a serial number which is used as a primary key, that's usable. We'll call that field 'serialNumber' in our example.
• Define your invoiceNumber field as a text or number field with this auto-enter calculation: If( Get( MultiUserState ) = 2; serialNumber; "" ). Be sure to leave the box checked that says 'Do not replace existing value of field'. This will leave the invoice number blank when working offline, but fill it in when connected directly to the server.
• In the primary key selection screen in MirrorSync, select invoiceNumber as the write-back field. This will cause MirrorSync to write invoiceNumber from the server back to the spoke file when the record is first synced.
• Your choice of MirrorSync-managed or Developer-managed is unaffected by this setting; pick appropriately depending on whether you are using serial numbers or UUIDs as your primary key.

MIRRORSYNC_DATABASEID for user-visible numbers

Let's say that you have a single serial number as your primary key, which is doing double-duty as an invoice number field. You can use the $$MIRRORSYNC_DATABASEID global variable to help. Using this approach, MirrorSync ensures that each file being synced gets assigned a unique sequential number, starting from 1. You can use this number in conjunction with a traditional serial number to create a unique number than can be used as a user-visible field as well as a primary key. The serial numbers will either be text (i.e. "1.1", "1.2", "1.3"…) or numeric (10,001 for user 1, 20,001 for user 2, 30,001 for user 3…).

Advantages of this approach:

• Reasonably short user-visible numbers
• IDs are assigned immediately upon record creation, without needing to wait for syncing

Disadvantages:

• spoke users must do an initial sync before they can create records.
• Requires a startup script to run, which means it won't work with custom web publishing applications.
• IDs are not sequential, so you can't assume that an ID with a higher value was created before or after another ID.
• If you are using numeric version, limits users to creating a fixed number of records before they start conflicting.
• If you are using text version, you must switch all primary key and foreign fields from 'number' type to 'text.'
• Setup is a bit more complex than write-back fields.

To use this approach, follow these steps:

• Set the 'MirrorSync setup' script as your startup script. If you already have a startup script, call the 'MirrorSync setup' script from your existing script. This will set the $$MIRRORSYNC_DATABASEID global variable. This clientId field is stored in the MirrorSync table, and is assigned when the user does their first sync. Each synced database will get a unique, sequential clientId. Manually run the script now to set that global variable before proceeding.
• We assume you have a serial field called 'serialField', and that your relationship graph uses this field as a key field. Start off by duplicating this field, to create a new field called 'serialField copy.' Do NOT place 'serialField copy' onto the sync layout for the table.
• Change serialField to an auto-enter calc. Uncheck the box that says 'Do not replace existing value'. Also make sure that the 'Prohibit modification of value during data entry' is unchecked. Set the formula to one of the following:

For text keys, use this formula. This will result in primary keys that look like "1.1", "1.2", "1.3" from device 1, and "2.1", "2.2", "2.3" from device 2. Feel free to modify as desired. Be sure to change the field type from number to text. Also change all foreign keys that relate to it to text.

$$MIRRORSYNC_DATABASEID & "." & serialField copy

For numeric keys, use this formula. This will result in primary keys that look like "10001", "10002", "10003" from device 1, and "20001", "20002", "20003" from device 2. Replace whatever number you want instead of 10,000 to give a wider or narrower range of numbers.

$$MIRRORSYNC_DATABASEID * 10000 + serialField copy

• For primary key configuration in MirrorSync, select Developer-managed, and set 'serialField' as your primary key.

Write-back values for SalesForce-to-FileMaker configurations

This section pertains to users developing a SalesForce-to-FileMaker configuration. If you're setting up an XML configuration(MS 5 or earlier) or a MirrorSync 6+ configuration with SalesForce as the hub and FileMaker as the spoke, you can write data generated by Salesforce back to the corresponding FileMaker record during a sync---like the auto-generated SalesForce primary key. To do this, follow the steps below:

• Step through the sync configuration to the page where you specify the primary keys and timestamp fields
• Select the SalesForce field you want to write to FileMaker as the hub-side write-back value
• Select the FileMaker field receiving the SalesForce field value as the spoke-side write-back value

How do I configure and use two foreign keys as a primary key?

This is a common configuration for join tables in many-to-many relationships. In the primary key configuration screen, use the drop-down menus to select the two foreign keys.

Please keep in mind primary keys need to be unique. If using two foreign keys, there may be an instance where an error occurs about a duplicate node ID, since FileMaker is not validating the fields together and making sure they are always unique in a table. To avoid this, make sure:

1. Ensure that the combination of two foreign keys only ever occurs once in the database.
2. If you need to be able to have multiple join table records with the same foreign keys, add a regular serial number field to the join table and use that as the primary key, instead of the compound foreign keys.
3. The foreign key fields must both have the 'Not empty' validation in order to appear in the pull-down menu of eligible fields. If your join table needs to have foreign keys that may be empty, you should add a traditional single primary key to the table instead of using compound foreign keys as an identifier for MirrorSync.

Why does MirrorSync need to know about foreign keys?

MirrorSync uses foreign keys for two purposes. If you are using MirrorSync-managed primary keys, then it needs to know about the foreign keys because it needs to rewrite them when it writes between databases see the Primary Key section above.

In addition, even if you're using developer-managed primary keys, such as UUIDs, MirrorSync needs to know about relationships so that it can insert/update parent tables before children tables. This allows validation rules that check referential integrity to work correctly. MirrorSync deletes records in reverse order from insertions (delete children, then parents) in order to avoid triggering cascading deletions, which are less efficient.

MirrorSync 2 adds a new feature that automatically detects foreign keys, if you are using FileMaker 12 or later, so this step should not add any time to the configuration process. If you are using FileMaker 11 AND you are using developer-managed primary keys AND if you do not have any referential integrity validation making sure that foreign keys point to valid parent records, you can safely skip this step.

Configuration questions

What type of sync configurations are supported by MirrorSync?

There are a lot! MirrorSync can sync between any combination of FileMaker Pro/Go, FileMaker Server, MySQL, SQL Server, Oracle, Salesforce, Amazon RedShift, and any JDBC-compatible database, such as PostgreSQL. Here is the complete list of supported configurations:

• FileMaker Pro or Go syncing with FileMaker Server
• FileMaker Pro or Go syncing directly with MySQL / Oracle / SQL Server / JDBC database / Salesforce / RedShift (in this configuration, FileMaker Server is needed during sync setup, but not for actual syncing)
• FileMaker Server syncing with MySQL / Oracle / SQL Server / JDBC database / Salesforce / RedShift
• FileMaker Server syncing with another FileMaker Server
• MySQL / Oracle / SQL Server / JDBC database / Salesforce / RedShift syncing with another MySQL / Oracle / SQL Server / JDBC database / Salesforce / RedShift

In all these configurations, one database is designated as the 'hub', and one or more other databases are called the 'spokes.' The hub database is where the data starts when configuring MirrorSync, and the spoke database(s) will be set up to match the hub during the initial sync. There can be multiple spokes, although they must be the same type of database. Here are some examples of multi-spoke configurations:

• FileMaker Server hub with many FileMaker Go spokes
• FileMaker Server hub with many FileMaker Server spokes
• FileMaker Server hub with many MySQL spokes
• MySQL hub with many FileMaker Server spokes
• Oracle hub with many MySQL spokes

MirrorSync configurations with SQL Server require Java 8 or above. MirrorSync does NOT support syncing FileMaker Pro/Go directly with other FileMaker Pro/Go devices without using some database such as FileMaker Server as the hub. It also does not support different types of spoke databases, such as syncing a FileMaker Server hub with a MySQL spoke and an Oracle spoke.

If you need to sync with a database or API that is not on this list, contact us and we'll get you a quote to support it. We've created custom implementations for state governments that use MirrorSync to sync with Web XML API's. We've also synced FileMaker Server with iTunes libraries for music companies. We're always open to a new challenge!

Should I use JDBC or XML to sync with FileMaker Server?

MirrorSync 1 and 2 used the XML Web Publishing Engine to read and write from FileMaker Server. Support for syncing using FileMaker's JDBC driver was added in MirrorSync 3. This is a major feature, because JDBC is significantly faster than XML in most, but not all, cases. It also supports record streaming, which means that while it's still possible to run out of memory during a very large sync, it won't be due to the amount of data stored in the records. It is also significantly more efficient for transferring container data. For all these reasons, we recommend that most users select the JDBC option.

However, there are still valid reasons to select the XML option:

1. If you want to customize the sync to only sync certain records for certain users, this can be done with JDBC, but it requires you to write an SQL qualifier. XML, on the other hand, works using traditional FileMaker scripting. If you prefer to use traditional FileMaker find requests, then you should select XML (See 'How Can I Filter Records Using SQL?' for some helpful tips).
2. If you want to trigger custom actions to happen when a record is inserted, deleted, or updated, this cannot be done using JDBC, and will require XML instead with the MirrorSync Customization script.
3. If MirrorSync is not on the same LAN as your FileMaker Server, for instance when doing server-to-server syncs between data centers, JDBC tends to be slow. Try both options to see which one is faster.
4. When using SQL qualifiers with large record sets (i.e. larger than 10,000 records or so), JDBC can often be slower than XML. Try XML to see if it's faster.

Does MirrorSync support FileMaker Cloud?

Yes, MirrorSync is compatible with FileMaker Cloud, but it is a very different architecture than regular FileMaker Server. In particular, you cannot install software on it, and there is no XML Web Publishing Engine. These are important differences to be aware of when using MirrorSync with FileMaker Cloud:

• MirrorSync must run on a separate computer from your FileMaker Server. 360Works offers shared MirrorSync hosting for $29/month which can communicate with FileMaker Cloud running on your server. You can also get a list of other approved hosting providers from our documentation. If you know how to set up Tomcat using SSH on Linux, you can run MirrorSync on your own Linux box on AWS.
• Since MirrorSync is not running on your FileMaker Server, you will not be able to use the download link feature to get copies of your database. Use the FileMaker Admin Console to get a spoke copy of the database to distribute to your users.
• Since all files are required to be encrypted on the server, and since you will be syncing with spoke copies of these files, all of your syncing users will need to know your server encryption password.
• The setup process requires you to run the MirrorSync setup script at various points in the process, in order for MirrorSync to retrieve metadata that it would normally get via the XML Web Publishing Engine. If you're used to setting up MirrorSync on regular FileMaker Server, be sure to read the instructions carefully to know when to run this script.

Does MirrorSync need the databases being synced to be identical?

No, not in MirrorSync 2 or later. MirrorSync is able to sync between databases with completely different tables and fields from each other. In addition, it can sync just a subset of records, fields, and tables. If you are syncing different databases, for example a dedicated mobile file with a much larger server database, select the option 'Sync with a separate mobile file' (for FileMaker Go) or 'Sync with a separate server file' (for server-to-server sync). With this option selected, MirrorSync will allow you to match up your layout and field names with a simple drag and drop interface.

I'm not seeing my databases from the Choose database button. What's happening?

Make sure XML publishing is enabled for the account you are using in MirrorSync. Also be sure the database is accessible from the machine you are installing MirrorSync on.

To test the XML Web Publishing Engine, try going to the URL http://yourServer/fmi/xml/FMPXMLRESULT.xml?-dbnames. You should see an XML document listing the databases in FileMaker Server. If you do not, or find a 404 or 401 error, then the XML Web Publishing Engine may not be configured correctly. Contact FileMaker tech support for help getting this running.

If there is an authentication required at that test URL, try disabling IIS authentication. FileMaker Server authenticates its password-protected databases; however, this method disables the IIS authentication layer. This will also disable authentication for other websites that are using IIS on that server.

1. From the Control panel, navigate to Administrative Tools > Internet Information Services (IIS) Manager
2. Select the website in IIS and choose Action > Properties
3. Navigate to the Directory Security pane, and select Edit for authentications. This button may differ in various Windows versions.
4. In the Authentications Methods pop up:
   • Make sure Anonymous Access is enabled
   • For Authenticated access, disable the authentication methods.
5. Press OK

I See a Filtered List of Databases, or No Databases

By default, FileMaker Server has the 'Filter databases in Client Application' setting enabled; this will filter out databases that FileMaker deems the user doesn't have access to; this can limit the databases that the user can select from the configuration wizard. Disabling this setting in your FileMaker Server Admin Console will fix this.

Configuration without FileMaker Pro Advanced

Since FileMaker Pro does not have the ability to copy and paste tables, you'll need to follow these instructions to create the MirrorSync table.

1. Locate the XML schema that came with your MirrorSync download.
2. Open FileMaker Pro and navigate to File -> Import Records -> XML Data Source.
3. In the "Specify XML and XSL Options" window, choose the "File" radio button, then click "Specify...".
4. Navigate to your MirrorSync download folder and choose the MirrorSync.xml file from the XML Schema subfolder.
5. Click continue to bring up the Import Field Mapping window.
6. Open the "Target" drop down menu, and select "New Table ("MirrorSync")"
7. Verify that the source fields and target fields have been matched correctly.
8. Click import.

After the MirrorSync table is created, you'll need to modify some of the fields:

Field	Options
id	serial number, with modification prohibited
modstamp	modification timestamp, with modification prohibited
lastErrorMessage	globally stored
webServerResponse	globally stored
_gSync1	globally stored
_gSync2	globally stored
_gSync3	globally stored
_gSync4	globally stored
_gLastInsertTable	globally stored
_gLastInsertResult	globally stored
container	Container data type

To achieve this, follow these instructions:

1. Navigate to File -> Manage -> Database...
2. Navigate to the "Fields" tab, highlight the id field and click the "Options..." button.
3. Check the box next to "Serial Number." Set it to generate on creation.
4. Check the box next to "Prohibit modification of value during data entry."
5. Navigate to the "Validation" tab and check the boxes next to "Not Empty" and "Unique Value" then press OK.
6. Next, highlight the modstamp field and click the "Options..." button.
7. Check the box next to "Modification" and ensure that the drop-down menu says "Timestamp (Date and Time)."
8. Check the box next to "Prohibit modification of value during data entry."
9. Navigate to the "Validation" tab and check the box next to "Not Empty" then press OK
10. Next, highlight the lastErrorMessage field and click the "Options..." button.
11. Navigate to the "Storage" tab and check the box next to global storage and press OK.
12. Repeat the last two steps to make webServerResponse, _gSync1, _gSync2, _gSync3, _gSync4, _gLastInsertTable, and _gLastInsertResult all be global fields.
13. Finally, highlight the container field, and select "Container" from the "Type" drop down menu, next to the "Options..." button. You are now finished manually configuring your MirrorSync table. Click the OK button to close the database management window, and return to the MirrorSync configuration. (Be sure to save your changes!)

Separation Model / Multiple-File Solutions

MirrorSync will work fine with multi-file solutions. You need to make sure that one of the files in your solution has a reference to every table that you want to sync. When using the MirrorSync download feature, be sure to select the multi-file option so that your spoke users will receive all the files for the solution.

If you are using the separation model with a UI and data file, configure MirrorSync to use the UI file. After the setup is complete, copy the MirrorSync table from the UI file to the data file, update the relationship graph to point to the table in the data file, and then delete the table from the UI file. This way, you'll have all the scripts and layouts in the UI file, while the MirrorSync internal metadata will be stored in your data file.

How do I sync External SQL Source (ESS) tables?

MirrorSync supports ESS tables, but it's very slow compared to using MirrorSync to simply sync the database from the external SQL database into your FileMaker Server. For this reason, we do not recommend ESS unless syncing is not an option for some reason.

If you would like to use ESS tables, there are additional steps. First, set up sync layouts and primary keys, modification timestamps, and creation timestamps. Next, make a duplicate of your file and host it on FileMaker Server. Make the following changes to that duplicated file ONLY. We'll need to remove the ESS references to allow the user to access the data offline by turning the shadow tables into regular tables.

1. First, delete the data source. This is vital to preventing duplicate records and other artifacts of using ESS. Open File > Manage > External Data Sources and delete the OBDC data source.
2. Next, open the File > Manage > Database and navigate to the Tables tab. Copy any ESS tables. Then delete them. Make sure NOT to delete the table occurrences. After all the italicized tables are gone, paste the tables back in. This ensures they are local FileMaker files, not references to external data.
3. Make sure the primary keys, modification timestamps, and creation timestamps are correct. The primary key needs to be an auto-entered serial number or generated UUID.
4. Open the relationships tab in the File > Manage > Database and relink the missing tables.

Now you're ready to configure MirrorSync! Start the configuration process, and select "Sync with a separate mobile file" on the third screen. Choose the file that was duplicated and modified, and continue with configuration as normal.

Why use MirrorSync instead of ESS?

Before configuring MirrorSync to sync between FileMaker Server and some SQL database, such as MySQL, you should consider whether using FileMaker's External SQL Source (ESS) is a better option for you. ESS allows you to access SQL tables directly from within FileMaker, as if they were local FileMaker tables.

Some advantages of using ESS:

• It is built into FileMaker and FileMaker Server, therefore there is no cost, although there is often a cost for the ODBC drivers to access the SQL database.
• There is somewhat less setup for ESS than MirrorSync, although you still need to install ODBC drivers and configure the ODBC DSN on the server. Using MirrorSync's ability to create FileMaker tables (this is done in the last page of the MirrorSync configuration process), the difference in setup time is not that significant.
• With ESS, there is less work when fields are added, deleted, or changed; go into field definitions and click the 'sync' button. With MirrorSync, you need to walk through the configuration process again so MirrorSync can get an updated list of fields.

Using MirrorSync, the SQL data is synced to local tables in your FileMaker Server. This gives you your own copy of the data that always matches what is in the SQL database. Advantages of this approach:

• It is much, much faster than using ESS. For doing reports on large tables, it is literally hundreds of times faster. For instance, we used to use ESS for pulling sales reports from our MySQL online store database. It took 40 minutes to pull a report for the quarter. By switching to MirrorSync and pulling the same reports with local FileMaker tables, it takes less than 10 seconds (240 times faster).
• Having local tables allows you to access and create field indexes, which is necessary if you want to use value lists, unique validation, or the 'paste from index' feature. It is also necessary for relationships that are not based on primary keys.
• Sorting records on local FileMaker tables is much faster than sorted records via ESS. Sorting more than a few thousand records from ESS becomes unusably slow.
• You can add your own additional fields to local tables (with ESS, you can add unstored calculations and summary fields, but nothing else)
• You can access data in your local tables regardless of whether your internet connection is working or not.
• You can rename the tables and fields in your local copy and use MirrorSync's mapping feature to match them up with the ESS database.
• ESS does not work with existing FileMaker tables. MirrorSync allows you to sync existing tables with the SQL database.
• Unless you configure an ODBC DSN on everybody's laptop (not an option for iOS), ESS will not work if you make a local copy of the database to your own laptop. With MirrorSync, you can copy the database from the server to your own computers and iOS devices and keep all the data.
• MirrorSync fully supports binary data in BLOB / Container fields. ESS does not.
• Local tables fully support multi-user record locking and broadcast notifications when records are changed. ESS does not.
• MirrorSync allows a developer to write custom SQL queries to only retrieve certain records. ESS does not (although a DBA could create an SQL view to solve this problem).
• MirrorSync allows custom SQL statements for generating primary keys for new records inserted into SQL. ESS does not.

In summary, ESS may be a good choice when you have a small amount of data which is accessed infrequently with simple list and detail views. MirrorSync is a better choice for data that is large, mission-critical, and accessed often.

Can I sync the same database server-to-server, and also server-to-client?

Yes, MirrorSync 3 and later have the ability to configure multiple sync configurations in a database, so you could have one configuration for syncing iPads with FileMaker Server, and another configuration for syncing the same database on FileMaker Server with MySQL, for instance.

Can I have different sync scripts to sync different tables?

Yes. Since MirrorSync 3 and later have the ability to have multiple configurations per database, you can do this. For instance, you could have a 'full sync' which syncs all tables in both directions, and then a 'save work order' sync which would only sync a handful of tables one-way from the spoke device to the server. You could then attach the 'save work order' script, which would be very fast, to a script trigger that runs whenever a work order is completed. If you set up multiple configurations syncing to the same hub database, be sure to use the same server address for both configurations (i.e. don't use an IP address for one of them and a hostname for the other).

Can I receive notifications of sync problems?

Yes, you can. There are two different settings related to this.

One is the admin e-mail address, which is configured in the MirrorSync setup process. By entering an e-mail address here and specify a severity from the pull-down menu, you can receive an e-mail message when certain events occur. You can select to receive e-mails every time a sync is done, only when there is a warning, only when there is a fatal error, and you can also specify whether you'd like to receive detail information with the affected primary keys.

The second setting related to this is in the MirrorSync.xml file. By modifying this file with a text editor, you can set the 'supportEmailAddress' field. When this is set, any bugs reported by users will be cc'd to this e-mail address, and tech support responses from 360Works will go to this e-mail address instead of back to the user. This allows your users to report any problems they encounter, while letting you (the developer) make any necessary technical adjustments.

Can I see performance statistics for sync activity?

Use the advanced notifications feature to accomplish this. The advanced notification button appears on the same configuration page where you set the administrator email address for notifications. When this is enabled, MirrorSync uses Amazon Web Services (AWS) to send notifications of sync statistics or failures.

• AWS Access key / AWS secret key: This is needed for all advanced notifications. If you don't have an AWS account, you can create a new one for free at this link. If you have an AWS account and you're not sure how to get an access key or secret key, follow the documentation at this link to authenticate as yourself. For better security, you can create a separate user with a custom policy that only allows SNS Publish and CloudWatch PutMetricData.
• AWS Region: Pick whichever geographic location is physically closest to your MirrorSync server, unless you have a reason to pick a specific region. When you log into AWS to see and configure notifications, you'll need to navigate to that region in the AWS console.
• Sync failure SNS topic: This is not a required field. If you're familiar with how to use SNS (Simple Notification Service), you can create an SNS topic and enter the Amazon Resource Name (ARN) in this field to receive a message whenever a sync fails. SNS topics can do whatever e-mail can, but they can also send SMS messages, or initiate custom code that you write yourself using AWS Lambda.
• Enable CloudWatch monitoring: This lets you send very detailed statistics to AWS CloudWatch. Check the boxes for all the stats that you're interested in tracking. Each checkbox will incur a monthly charge of $0.30 - $0.50. Once you've started running syncs with this option enabled, you can use AWS CloudWatch to view this data. You can also set up dashboards and alarms to easily visualize patterns in sync data, or get notifications if certain stats are above or below a threshold.

Can I receive notifications if my auto-scheduled sync stops running?

Yes. Refer to the previous question on performance statistics and configure CloudWatch. Then, set up an alarm in CloudWatch based on some metric that you're reporting, such as total duration. It doesn't matter very much which metric you pick, or what your alarm settings are. The important part is to configure CloudWatch to notify you if the alarm goes into an INSUFFICIENT_DATA state, which is what will happen if the auto-sync stops running.

Are repeating fields supported?

Yes, repeating fields are supported. However, each repetition in a field counts as 2 slots (there is a maximum of 998 'slots', or fields, per table). For instance, if you had an invoice table that had a field for customer, invoice number, line item description[100 repetitions], and line item unit price [100 repetitions], that would count as 402 slots (1 for customer, 1 for invoice number, 100 * 2 for descriptions, and 100 * 2 for unit price). Be careful about this, because it's easy to exceed the 998 slot limit with repeating fields.

Setting up a SalesForce to FileMaker Server Configuration

Setting up a MirrorSync configuration with SalesForce and FileMaker Server is mostly a straightforward process, and is very similar to a standard FMS to FMS configuration. However, there are some caveats you should be aware of, and some recommended approaches to help you navigate this process as Salesforce has unique behaviors regarding data access and entry.

You will need your Salesforce API token. If you do not have one, you can retrieve one through the Salesforce Settings page via the Reset My Security Token option.

Although the names for the fields in Salesforce may not be the exact same between the web interface, the documentation, and the API call return values, they are similar enough as to be identifiable(ex. table 'Products' is called 'Product2' in the documentation, and called 'Product' in the list of available standard objects returned by the API). The easiest approach may be to create fields in FileMaker based on the field definitions provided in the Salesforce developer documentation. A link to their table and field definition documentation is provided below.

Standard Objects(tables)

MirrorSync communicates with Salesforce via the SOAP API, so make sure to reference the appropriate Salesforce API documentation. You may also want to match your FileMaker db schema with that of Salesforce so that relationships are properly recognized in the sync. A link to docs on Salesforce schemas is provided below:

Data Models(schema)

We recommend setting up a sync configuration with Salesforce as the hub. This is because Salesforce has a lot of restrictions regarding field access and mutability. Most of the restrictions are already applied by MirrorSync in the configuration setup process, and are more transparent using that approach.

In general, if the field is editable in Salesforce(Classic or Lightning interface), MirrorSync is able to update/write to the field, and for fields not editable in Salesforce, MirrorSync likely will not be able to update/write to the field. There are fields accessible by MirrorSync that can be written from Salesforce to FileMaker, and only from Salesforce to FileMaker. You can still select a table to employ a bidirectional sync, and individual fields within the table can be set to sync one-way. For many fields in Salesforce, a one-way sync setting will be applied automatically by MirrorSync as it can detect which fields are read-only.

If you matched the field definitions in FileMaker as they are specified in the Salesforce documentation, you can just step through the MirrorSync configuration and map the tables and fields accordingly.

You will need separate keys for FM and Salesforce, as Salesforce writes its own keys. This also means you must employ MirrorSync-managed keys in your sync configuration. We recommend setting up write-back fields in FileMaker to receive the Salesforce pk. They will be of type Text.

There are a few niche, special cases that you should be aware of during the setup process. Some of these cases you should look out for include:

• By default, system fields like CreatedById and CreatedDate are not writeable. You can follow Salesforce documentation to enable that capability.
• Salesforce requires that there always exist a Pricebook record, Standard. This record cannot from Salesforce so if you set up a one-way FileMaker-to-Salesforce sync config, you will not be able to 'delete the spoke' if syncing this table.
• The Salesforce object Asset requires values in either or both of the following foreign key fields: AccountId, ContactId.
• Write-back Values

Video Demonstration of SalesForce to FMS Configuration

(Coming Soon) If you have any specific SalesForce to FMS Configuration questions, please send in an email to support@360Works.com, and we will try to answer them promptly. Thanks.

Configuration Maintenance

Each configuration in your list has a right click menu for additional maintenance procedures (context menu pictured below). Many of these options are self-explanatory, the impact and usage of the others are documented below.

Configuration Context Menu

What can I do if my SyncData is corrupt?

MirrorSync keeps an internal record of the sync activity, including primary keys, foreign keys, creation date, mod count etc. All this data is stored in the SyncData folder. Sometimes, for a number of reasons, the SyncData may become corrupt. If this happens there are 2 options in order to resolve the issue. We recommend first trying to Repair SyncData and if that does not fix your issue you can Reset SyncData. Both of these operations are initiated in the MirrorSync Configuration Client.

Repair SyncData

Repairing the SyncData will try to rebuild the internal database for MirrorSync. It can take a few seconds for small databases, or hours for large databases, and no syncing can happen while this is running. Existing spoke files can continue to be used after this finishes.

Reset SyncData

If Repair SyncData did not work, your next step to take would be to Reset SyncData. Resetting the SyncData will delete the SyncData on the server for the specified configuration, and any other configurations sharing the same hub database. You can try syncing existing spoke files, however, they will take longer on their next sync and may require conflict resolution for recently changed records. We recommend if you Reset SyncData you re-distribute spoke copies again, this will provide the best opportunity to not corrupt the SyncData again. If it is a Server-to-Server sync you will be prompted for how you would like to perform the initial sync after resetting the sync data and hitting Sync now. Delete spoke & Rebuild sync data. Rebuilding the sync data will examine the databases record-by-record since a date you specify to find changes between them. This could take a long time for large databases. Delete spoke will delete all records from the spoke database and then be re-inserted from the hub. This cannot be undone, please contact 360Works with any questions before moving forward with Reset SyncData.

Reset SyncData (only AuditLog)

MirrorSync's sync data consists of two main parts, the "main" sync data and the "AuditLog" sync data. AuditLog sync data is usually much larger, and also much less critical to sync operations. The AuditLog sync data is used for only one thing: Merging field-level conflicts. If you reset the AuditLog sync data, changes made before the audit log was reset will not be able to be merged with changes made after the audit log was reset. This is usually not a big problem, making it reasonably safe to periodically clear out your AuditLog sync data to clear up space. It can also help to reset the audit log SyncData if you're getting sync error messages similar to "Conflict was not resolved"

Clear configuration cache

This option is rarely necessary. After selecting this option, the next sync for each device will be slower than normal while MirrorSync checks all field names and types. MirrorSync usually detects these changes without any admin intervention, but this step is occasionally needed if MirrorSync 'misses' a change.

Compact SyncData

This will compress your SyncData, reducing the size on disk. In MirrorSync 4 and earlier, this would often free up significant space, but in MirrorSync 5 and later, the process for writing SyncData is much more efficient, so it's unlikely that running this operation will be beneficial. Sync operations cannot run while this operation is in progress, and it can take a long time for SyncData larger than 10 gigabytes, so while it is never harmful, we don't encourage using it.

Re-sync old records

MirrorSync normally ignores changes older than the last successful sync. There may be times, however, when you'd like to treat older records as if they were recently modified and re-sync them. For instance, if records were not written due to validation failures, and then you modified the validation rules to allow those records to sync, you could use this function to retry those old records that previously failed to write.

Scan for invalid data

This option will scan your database(s) for problems that may cause a sync to fail, including:

• Blank primary keys
• Duplicate primary keys
• Invalid timestamps
• Invalid container data
• Large text lengths. FileMaker's JDBC driver fails when trying to sync text fields with greater than 65,535 characters or longer. If you are getting this error you can use this option to scan all your syncing tables for fields with text greater than 65,535. You can then shrink the text to be less than 65,535 characters or remove it from the sync completely. If neither option is ideal, you can switch to XML to bypass this JDBC bug.

It's a great idea to use this function before your first sync, or any time you get an error message that MirrorSync received a '?' instead of a valid result. If MirrorSync uses the ExecuteSQL function to fetch a batch of data that contains even a single invalid timestamp field value, this will cause the ExecuteSQL function to return a '?' for the entire batch. In addition, the invalid data scan will look for duplicate or blank primary keys.

Note:

A successful scan doesn't indicate that a sync will always succeed, and while MirrorSync will identify many issues, won't correct them for you as many of these problems may have different solutions depending on your preferences. However, if you get an error message related to invalid data it's a good idea to run this scan to get a more comprehensive view of what's happening.

Repair SyncData

The internal MirrorSync SyncData can occasionally become corrupted. The most common cause of this is running out of disk space or RAM in the middle of a sync. If corruption is detected, an email is automatically sent to the 360Works support team. We will notify you of this, and in many (but not all) cases, repairing the sync data can fix the corruption.

Disk space usage

Following is a description of files that MirrorSync writes to disk. You'll see several references to the MirrorSync.xml file below, see the provided link for details. If you're on a Mac, and you want log files to go on an external volume, you'll need to grant special OS permission to allow MirrorSync to do this. See 'Integrating MirrorSync with External Volumes on macOS' for more details.

1. The core sync data. This is critical for the operation of the sync. It grows (never shrinks) until the sync data is reset, which empties it out completely. It doesn't usually grow at an extreme rate, and its size is basically a function of the number of total records across all tables multiplied by the number of spoke copies that have been synced. The amount of data stored in each record doesn't change the size of this file. You can reset the sync data by right-clicking the name of the configuration and selecting "Reset SyncData -> All SyncData." This should ideally be accompanied by distributing new spoke files to users. It is stored on whichever volume MirrorSync is installed, although its location can be customized by editing the '360WorksDirectory' setting in the MirrorSync.xml file.
2. The audit log database. This also grows until sync data is reset. It can often grow faster than the main sync data, depending on how often the same record is changed and the amount of data stored in the records. It is much less disruptive to periodically manually clean out this database by right-clicking the name of the configuration and selecting "Reset SyncData -> Only Audit Log". The only downside of doing this is that MS will not be able to do field-level conflict merging until it builds the audit log back up from syncing records. Changing the '360WorksDirectory' setting in the MirrorSync.xml file will change where this data is saved.
3. The MirrorSync logs. These are needed primarily for bug reporting and troubleshooting. Other than that, it's safe to throw them away. They are typically 5-100 megabytes per day, but they can grow very large (like 5 gigabytes per day) if there is a very large amount of sync activity. They are kept by default for 14 days, and are kept in C:\Program Files\360Works\Applications\logs. You can reduce the number of days they are kept by modifying the 'logRetentionDays' setting in MirrorSync.xml. You can also change where they are stored by modifying the 'logDir' parameter in the MirrorSync.xml file. Also, see 'storing logs on an external volume' below
4. The MirrorSync audit log text files are kept in the same directory as the MirrorSync logs. Unlike the audit log database mentioned in #2, these are human-readable text files that list every write operation that MirrorSync has ever done. They are automatically cleared out after the same amount of time as the MirrorSync logs (#3 above). They do not typically get unreasonably large. If you ever wonder whether MirrorSync changed or deleted a record, you can look at these text files to see that. Like the log files in #3, you can change where they are stored by modifying the 'logDir' parameter in the MirrorSync.xml file.
5. Temp files. These are created temporarily when downloading spoke copies, syncing container data, and syncing large amounts of non-container data. They are usually cleared out after a sync finishes. The temp files will be kept on the same volume where MirrorSync is installed. In some cases (like making spoke copies of very large databases), there might be too much data to store on this volume. You can customize the 'tempFilesDir' in MirrorSync.xml to override where these files are stored. All files in /Library/360Works/Applications/temp are safe to delete if storage space becomes an issue.
6. Configuration backups - every time a MirrorSync configuration is changed, all the previous configurations are saved in a file named 'MirrorSync_configsAndSchedules-XXXX.dat', in the same directory as the core sync data. If you ever make a big mistake and need to recover your last configuration, you can stop MirrorSync, pick the appropriate backup file based on the date, and rename it to remove the -XXXX portion. Start MirrorSync back up and it will load the configurations from that file. It is safe to delete any of these old backups, although they don't take up much space.

Storing logs on an external volume

MirrorSync logs on a busy server can get very large. We sometimes see log files of 3 gigabytes per day. To prevent your boot drive from filling up with log files, if you have a busy server, you might want to configure logs to be stored on a separate volume. On Mac or Linux, this can be done by creating a symbolic link from /Library/360Works/Applications/logs (Mac) or /var/log/tomcat9 (Linux) to the logs directory. For instance:

ln -s /logs/tomcat /var/log/tomcat9

On Windows, you cannot use aliases for this, but you can use a Directory Junction:

mklink /J C:\Program Files\360Works\Applications\logs {target-directory-on-external-drive}

Sync questions

Does MirrorSync work with timezones?

Yes and No.

For spoke users who are syncing, Yes. When MirrorSync gets a list of all modifications, it will automatically detect the spoke user's timezone and clock drift and compensate for it.

For a server-to-server sync, Yes. MirrorSync can detect FileMaker server's current time and compensate for it, just like it does for spoke users.

For non-syncing users connecting directly to FileMaker Server, No, not automatically. You'll need to perform a small workaround to make this work. It gets a little complicated, so an example makes it more clear. Let's say your FileMaker Server is hosted in New York. A user in New York runs a sync at 12:30 PM Eastern Time. 30 minutes later, a user in San Francisco modifies a record on the server at 1 PM Eastern time / 10 AM Pacific Time. Due to the way that FileMaker Server works, that modification timestamp on the server in New York will say 10 AM. It will not be converted to Eastern Time, nor will FileMaker Server store any time zone information for anybody to tell that it was actually representing Pacific time. Essentially, it's just an incorrect value. When the next sync runs, MirrorSync is going to request records modified since the last sync at 12:30 PM, which is going to miss the change made by the San Francisco user, even though it was made since the last sync.

The solution to this problem is very easy - it actually takes less time to do than it did to read that explanation! Let's say that your table contains a regular modification timestamp called 'Modification timestamp'. Create a new field called 'Host modification timestamp' (or whatever you prefer), and set it up as an auto-enter calculation field. Be SURE to uncheck the box titled 'Do not replace existing value of field (if any)'.: If the normal modification timestamp field in the current table is named "zzModificationTimestamp", your calculation for the host modification timestamp in that same table would be:

Let ( x = zzModificationTimestamp; Get(CurrentHostTimestamp ) )

If MirrorSync is set only to sync fields on the layout, don't forget to add it to your sync layout, and REMOVE the regular modification timestamp from the sync layout!

Use this field as the modification timestamp in the MirrorSync configuration process, instead of the regular modification timestamp.

This works by using the FileMaker Server clock, instead of the user's clock, for the modification timestamp. Now, when the San Francisco user modifies a record at 10 AM Pacific Time, FileMaker Server in New York will store 1 PM (the server's local time when that change was made) in the Host modification timestamp field, and MirrorSync will work with time zones correctly.

After you make this change, there is no need to populate the new host modification timestamp field with data.

Can I share a device among multiple users?

Yes, but…

If all users on that device log in with the same username, then it's probably OK, as long as you make sure that you are not using record level access privileges on the spoke device that would grant different record access to different users.

If you want each user who shares a device to log in with different usernames, then the solution is to put a separate copy of the file on the device, one for each user. Make sure that each user is only able to log in to the file that is assigned to them.

MirrorSync prevents multiple users with different usernames from sharing the same spoke file, because if record level access privileges or script filters were configured differently for each user, that would lead to the sync process incorrectly deleting the wrong records on the server, which would be very bad.

Does MirrorSync sync container fields?

Yes, MirrorSync works with FileMaker container fields. Container fields do take longer than other field types to synchronize, so remove them from your sync layouts if you don't need them to be synced.

MirrorSync supports external container fields (both Secure storage and Open storage), as well as regular (not external) container fields. It does not support referenced container fields (When the user checks the box 'Store only a reference to this file' when attaching the file)

What about SuperContainer? Does MirrorSync work with that?

Yes, it does. However, with SuperContainer the approach is very different. You will need to have a field on your sync layout that contains the SuperContainer URL that the web viewer is pointing to. That way only the URLs are synchronized, not the actual files - they remain on the SuperContainer server. The advantage is syncing is very fast - there is no binary data being transferred. The disadvantage is that you'll only have access to the files stored in SuperContainer when you have working network access from your computer or iOS device.

Does MirrorSync sync external container fields?

Yes, MirrorSync works with external container fields, assuming your FMS folder settings are configured correctly (see the warning below for details). There are no extra steps to take when syncing with FileMaker Pro or FileMaker Server. If you are using external container fields on an iOS device AND if you are syncing with a full copy of the database (as opposed to an empty clone), there are a few steps after configuration that are required.

1. Download a copy of the database from MirrorSync or FileMaker Admin
2. Open the copy on a computer, and navigate to File > Save a copy as...
3. Under the type drop down, choose "self-contained copy (single file)"

This ensures the copy of the file embeds the containers into the file for easy use with FileMaker Go and offline functionality.

Note:

There is an unfortunate behavior in FileMaker in this process: when saving the self-contained copy, FileMaker updates the modification timestamp for every record that has an external stored container field. That's not a problem normally, but when used with MirrorSync, it will make the initial sync very slow. The reason for this slowdown is due to MirrorSync assuming that all of those records have been modified on the client, so it writes all the server data to the client (including the container data).

The workaround for this is to uncheck the modification timestamp auto-entry before saving the self-contained copy, and then turn it back on after the copy has been saved. This modification timestamp behavior has been reported to FileMaker, Inc. and will hopefully be resolved soon.

Warning:

Certain FMS folder settings will keep the FileMaker backup process from including external container data with your backups, which means that your external container data will not be bundled with your MirrorSync client database download (or will not be pushed to the spoke server, in the case of a server-to-server sync). For more details, review this section of our documentation

Why doesn't my iPhone / iPad show a progress bar when I click the download link? Is anything happening?

When you create a download link to share with users, it creates a URL that users can click on to download a copy to their own device. For large databases and/or slow networks, this can take a very long time. With iOS 6, you got a little progress bar that fills across the tab until the file downloads. Unfortunately, iOS 7 removed this download progress indicator, so it may look like nothing is happening after you click the download link. There's nothing that can be done to fix this: we suggest filing a feedback suggestion with Apple directly or using a different browser (most other iOS browsers show a progress bar). If you're really convinced that the downloads are not working and you've waited a long time (sometimes up to an hour for very large files), send us a problem report using the link on the MirrorSync launch page. Be sure to tell us what time the download started so that we can find it in the log.

Does MirrorSync require users to sync with full access accounts?

No. MirrorSync requires a full access account only once: to paste in the table, scripts, and layout. When users run the sync, they will use their own user account privileges, with whatever access that grants them. See the 'Customizing MirrorSync' section below for tips on restricting user accounts.

If you are using field-level permissions, for example if you set certain fields to be read-only or no access, then that will cause a problem during syncing. The MirrorSync script running on the spoke device will not be able to write to the read-only field on the spoke file, since it runs with the same limited permission as the spoke user. To solve this problem, set the 'MirrorSync' and 'MirrorSync customization' scripts to run with full access. Do this on both the server file and the spoke copy. This will allow MirrorSync to read and write fields that the user themselves may not have read or write access to. If you do this, you will not be able to use record-level access privileges to restrict which records are synced with the spoke copy - use the MirrorSync customization script instead.

Does MirrorSync do conflict resolution?

Yes, MirrorSync has robust conflict detection and field-level merging (added in version 2). It will detect when the same record(s) are modified on both database and take action. There are two settings in the configuration client that will control how this works:

• Merge the changes together
  • If MirrorSync sees that all the changes made in the databases were to different fields (i.e. firstName on the server and lastName on the client), it will automatically merge the changes together. It will not tell the user that a conflict occurred, it will just write the merged record to both the server and the client. If any changes were made to the same field, it is treated as a conflict and not merged.
• Flag the edit as a conflict
  • MirrorSync will treat all modifications to the same record as a conflict, regardless of whether they were made to the same fields or different fields.

If a conflict occurs, then MirrorSync will resolve it based on how that table is configured to resolve conflicts. There are four options for conflict resolution:

• User picks
  • MirrorSync will present a web-based interface to the user, summarizing all conflicts and showing them exactly which fields were changed on both databases. It color-codes the actual sections of text that were changed, making it easy for even inexperienced users to make the best choice. The user is able to pick one entire record over another, pick individual fields from each record, and even manually edit the result to combine changes from both records. They also have the option to automatically select the most recent change (see next option).
• Most recent change wins
  • MirrorSync will pick whichever record was last modified as the winner of the conflict. If Joe changes a record on his iPad at 3:00 PM, Kate changes the same record on her iPhone at 3:30 PM, and Tom changes the record on the server at 4:00 PM, then when Joe or Kate sync their database, Tom's record will be selected as the winner, regardless of when Joe and Kate do their sync.
• Hub always wins
  • MirrorSync will always pick the value on the hub (typically FileMaker Server) as the conflict winner, regardless of other considerations. This is not generally recommended, because it is essentially first-sync wins behavior: Whoever syncs to the hub first will win conflicts with other spoke users who modify the same record and sync. This setting can be useful if changes made by users that are directly connected to the hub should be given priority over spoke users.
• Email administrator
  • This is very similar to 'User picks', except that instead of the user who is doing the sync deciding, an e-mail is sent to the administrator email address specified in the configuration. The administrator can resolve the conflict using the same web-based user interface described above, after which the user will be able to sync normally.

Is MirrorSync transactional? What happens if the connection is lost while syncing?

MirrorSync is not 'transactional' in the strictest sense. If the connection is dropped during a sync, it is possible for records from one table to be written to the server, while records from another related table may not be written. However, MirrorSync keeps track of which records have been written and which ones have not, and it will resume from where it was on the next sync and complete everything as if the first sync had finished. If a record on the server cannot be written because it is locked (being edited), the spoke user will get a warning error message telling them this. MirrorSync will continue to retry that edit operation each time the sync script is run.

Can I use external authentication with MirrorSync?

Yes, MirrorSync supports externally authenticated accounts with no modifications. So long as the username on the local copy matches the username on the server, you can set up one password for local access and another for external authentication with the server. The passwords do not need to match between the local file and the hosted file.

Does MirrorSync respect validation settings in FileMaker?

Yes, IF they are set to 'always' (as opposed to 'Only during data entry', which is the default). The 'Allow user to override during data entry' has no effect on whether MirrorSync respects validation. If you are using UUIDs for your primary keys, we recommend setting a 'Unique value' constraint on them with the 'Always' choice selected. This protects against duplicate records, which can happen if MirrorSync is mis-configured.

Does MirrorSync retain formatting on text fields in FileMaker?

No. The formatting metadata does not get pulled due to limitations in the FileMaker XML or JDBC data source. This means that text color, style, and font information will be stripped out and converted to plain text during the sync. Some customers have come up with their own workarounds using the GetAsCSS function and custom functions to translate formatted text to plain text and then back again, but this is not something that 360Works can assist with.

What if the sync takes longer than the auto-sync frequency?

The auto-sync frequency is the delay between the end of one sync and the start of the next sync. This means that if you sync generally takes 120 seconds, and your auto-sync frequency is set to 60 seconds, then the sync will run every 180 seconds. It will not be interrupted or canceled or run non-stop. The auto-sync feature is only available for server to server syncs.

Can I have more than one modified auto enter field on my sync layout?

No, MirrorSync only allows for one "modified" auto-enter field on the layout, the modified timestamp. That field does not get treated as a data modification by MirrorSync. MirrorSync will treat any other modified auto-enter field the same as it would any other data field on the sync layout. The reason why these are not allowed on the sync layout is because they will always result in a conflict. For example, User A and User B both sync a Contacts database at 12 PM. User A modifies the value in the "first name" field to "John" on a record in their file and User B modifies the value of the "last name" field to "Smith" on the same record in their file. At 1 PM user A syncs their file with the server. At 2 PM user B starts their sync. At this point, MirrorSync sees that User B last synced at 12 PM but the Server was updated at 1 PM by user A. MirrorSync sees that this could potentially cause a conflict. It then sees if these two changes can be merged. Since two different fields on the same record were modified, MirrorSync will merge the changes. However, if a "modified by" auto enter field had been on the layout, a conflict would happen because the "modified by" field will always be different between users and thus the conflict has to be resolved before the sync can continue. This will either result in a loss of data or a canceled sync depending on the conflict resolution strategy. This problem will happen anytime the same record is modified by different users.

My sync data is taking up a lot of hard drive space. Can I reduce the size?

Yes! MirrorSync 5 has ability to compact sync data. This can take quite some time with large data sets and no syncing can occur during the process so be sure to do this during a time when no syncs can potentially run. To compact sync data. Open the Config client and "right click" the sync you want to compact and choose "Compact Sync Data"

FileMaker Server 17 does not have XML Web Publishing automatically enabled. Do I have to enable it manually?

No, MirrorSync 5 will automatically enable XML Web Publishing on FileMaker Server 17. However, if you would like to manually enable it, you can follow the instructions in our Enable XML Web Publishing documentation.

Record could not be written because of an error 507: Value in field failed calculation test of validation entry option

This is an error coming from FileMaker. This means one of the fields you are trying to sync does not pass a field level validation by calculation, which will cause the modifications in the set to revert and not get updated. MirrorSync cannot get around field level validation. We recommend using Record level access protection based on the privilege set, this way when you sync with XML you have the ability to run as Full Access privilege.

If you have any Auto-enter Data fields (including Lookup fields) that have validation by calculation, these fields will also need to pass validation even when not included on the syncing layout. For example, if you have validation based on a field having a certain value, the validation will always fail if it does not pass that calculation regardless of whether it is included in the sync.

Server-to-server sync

How do I set up server-to-server syncing?

For the most part, the process for setting up server-to-client sync is almost identical to server-to-server sync. There is one difference to be aware of:

• In the spoke configuration, you select 'copy or clone of existing file' or 'separate file.' If the first option is selected, you don't actually put the file on the other server during the setup process - go through the setup process, and when you're finished, save a copy or clone from the hub server using the MirrorSync download feature and upload it to the spoke server. If you select 'separate file', then you will need to have the file up and running on the spoke server during configuration.

How do I run a server-to-server sync?

There are four ways that server-to-server syncs can be triggered:

• By using the 'Sync Now' button in the MirrorSync configuration client.
• By checking the 'Auto Sync' checkbox in the MirrorSync configuration client, and selecting a desired frequency in seconds. This will continue to run as a background service, even if the MirrorSync configuration client isn't running. (see important note below)
• By running the MirrorSync script in FileMaker Pro / Go which is connected as a guest of FileMaker Server.
• By submitting an HTTP request to the MirrorSync server. For details on how to do this, see the 'How can I trigger a Server-to-Server sync by URL?' section below.

Warning:

You should never use the auto-sync option for initial syncs. Most errors will occur on an initial sync, and if an error occurs during an initial sync it's almost always faster and safer to solve the problem in the error, replace the spoke with a copy of the hub again, reset the sync data using the config client, and restart the sync from the beginning. It's recommended that you right click your config and scan for invalid data before running an initial sync, to reduce issues that could force you to restart.

How can I be notified of a problem that occurs during server-to-server sync?

There is an administrative e-mail address that is set during MirrorSync configuration. If you set this, you will receive e-mail notifications whenever the sync fails (or succeeds, depending on what granularity you set admin e-mails to). These e-mails are sent via Amazon Web Services, and thus do not require you to configure an SMTP server.

How can I trigger a Server-to-Server sync by URL?

The URL will be formatted like this:

https://yourServerAddress/MirrorSync/triggerSync/*URL-encoded config name*?wait

So if your HUB server address is example.com and your configuration is named 'Inspection System', the URL you would need to initiate a Server-to-Server sync would be:

https://example.com/MirrorSync/triggerSync/Inspection%20System?wait

Performance questions

What is the performance like for server-to-server sync?

A real-life example is the 360Works online store database. We set up MirrorSync 3 to do a one-way sync from the live MySQL database to our FileMaker 15 Server. This database has 445,296 records across 20 tables.

Running an initial sync, which syncs all the MySQL data into an empty FileMaker database, took 1,612 seconds (27 minutes), which breaks down to an average of 276 records per second. This is 6.1 times faster than MirrorSync 2, which averaged 45 records per second for the same setup.

Incremental syncs run every minute and take 1-2 seconds.

What is the performance like when multiple clients are syncing simultaneously?

Very good. We set up a benchmark with 250 simultaneous syncing clients to test. You can view the results on the MirrorSync webpage.

How well does MirrorSync perform on slow networks?

MirrorSync is very efficient over slow networks, because it sends very little data other than the actual record changes, and it transmits this information in large chunks, which is more efficient. There are 2 HTTP request at the beginning of the sync, 1 at the end, and usually 1 HTTP request per table that contains any modifications. For example, if you have a 20 table solution, and there are changes in 5 of the tables, it will usually take 8 HTTP requests (2 + 5 + 1) to complete the sync. By comparison, the FileMaker home page takes 65 HTTP requests to load in a web browser.

What can I do to speed up syncing?

Before talking about ways to make MirrorSync faster, let's first talk about what type of operations take longest:

• For client-server-syncs (i.e. iPads syncing with FileMaker Server), the first initial sync for the first device is always the slowest, because MirrorSync needs to download metadata (primary key, modification timestamp, creation timestamp, record ID, record modification count) for every record in every table in the database. After that, any initial sync on any device will be much faster. If you reset the sync data (by right-clicking the configuration), or delete and re-create the configuration, MirrorSync will need to re-scan every record again.
• Initial syncs from a recent server copy are much faster than initial syncs from a copy made a long time ago.
• Of course, incremental syncs are faster than initial syncs.
• Any time a large number of records are written during a sync, the next sync on that device/database will take longer than usual.
• Inserts are generally faster than updates, especially for SQL databases like MySQL.
• MirrorSync can tell whether records are deleted. Checking to see if anything was deleted takes no extra time. However, if records were deleted, then MirrorSync scans the primary keys to find which record(s) were deleted, which does take longer.
• Container fields take much longer to sync than non-container data.
• The number of tables affects the speed of syncing: A good general rule is about .5-1 second of overhead per table, plus the time to actually sync changes.

MirrorSync is pretty fast without any special tweaking, but faster is always better. Here are some tips for squeezing out the best possible sync speed from MirrorSync:

• SSD drives: MirrorSync does many thousands of tiny read and write operations to its internal database when syncing large record batches. This type of usage will benefit greatly from the ultra-low latency of SSD drives. You will see a noticeable change in sync speed by installing MirrorSync on an SSD drive. You don't need to use an SSD for your entire hard drive - you can just put MirrorSync on the SSD by using a symbolic link in OS X or changing the installation location on Windows. The SSD performance benefits will improve all sync operations, but it will be most noticeable when doing very small syncs with few or no changes, and initial syncs for large database with hundreds of thousands of records.

• Tables with hundreds of fields can lead to slow syncs. Whenever a record is modified, MirrorSync writes all the fields for that record (not just the ones that were changed). Splitting these 'wide' tables into several 'narrow' tables will make your sync significantly faster, and will also make a big difference when connecting directly as a guest of FileMaker Server (when not using MirrorSync).

• Use FileMaker Server 14 or later, and experiment with both JDBC and XML for server communication. In most cases, JDBC is significantly faster, but there are exceptions where XML is faster. For instance, when the FileMaker Server is not on the same LAN as MirrorSync, XML is usually faster. Also, it is sometimes faster to use the MirrorSync customization script (which requires XML) than it is to use SQL filters (which use JDBC).

• When MirrorSync fetches changed records from FileMaker Server, it requests all changes since 10 minutes prior to the last sync. This 10 minute overlap compensates for users whose clocks are slow. If you're using the Host modification timestamp trick outlined in the timezone section above, then the user clock doesn't matter, and you don't need this 10 minute overlap. You can get rid of it by editing the MirrorSync.xml file and change the 'fmServerOverlapSeconds' parameter from 600 (10 minutes) to 5 (5 seconds).

• Use serial numbers instead of UUIDs, especially for large record sets with narrow tables. The difference between a 5 character serial number and a 36 character UUID adds up, especially when transferring large numbers of records that mostly consist of foreign keys, such as join tables. Don't make this change if you have a good reason for using UUIDs.

• Container fields hold lots of data and are sent individually instead of in batches like non-container data. If there are containers that you don't need to sync, remove them from the sync layouts.

• Container fields are transferred when any field on the same record is changed, even if the container has not been modified. To prevent unnecessary container syncing, move container fields to their own table, so that they will only be transferred when they are changed.

• If you want to just sync a subset of records, instead of the entire database, you can use either record level access privileges or scripted filters (see the customization section below for more details). Scripted filters are much, much faster than record level access privilege restrictions.

• If you're on FileMaker Server 10 or 11, switch to 12 or later to use XML, or 14 or later to use JDBC. The XML Web Publishing Engine was just not very fast in FileMaker Server 11. It is greatly improved in later versions of FileMaker Server. JDBC, which is usually faster than XML, requires FMS 14 or later.

• Speaking of the XML Web Publishing Engine, there is a configuration that causes a substantial slowdown when syncing with FileMaker Server 13 (this is not an issue with FileMaker Server 10, 11, or 12. It's also not an issue with FileMaker Server 14). This adds about a second onto every XML request, so requests that would normally take about 20 milliseconds wind up taking around 1,100 - 1,200 milliseconds. To fix this, open up the file at FileMaker Server/Web Publishing/publishing engine/jwpc-tomcat/conf/server.xml. Find this section:

<Connector URIEncoding="UTF-8" acceptCount="100"
address="127.0.0.1"
compressableMimeType="text/html,text/css,text/plain,application/javascript,application/json"
compression="on" compressionMinSize="2048" connectionTimeout="20000"
disableUploadTimeout="true" enableLookups="false" maxHttpHeaderSize="8192"
maxSpareThreads="75" maxThreads="150" minSpareThreads="5" port="16020"
protocol="HTTP/1.1" redirectPort="8443"/>

Remove the part that says address="127.0.0.1", so that it looks like this:

<Connector URIEncoding="UTF-8" acceptCount="100"
compressableMimeType="text/html,text/css,text/plain,application/javascript,application/json"
compression="on" compressionMinSize="2048" connectionTimeout="20000"
disableUploadTimeout="true" enableLookups="false" maxHttpHeaderSize="8192"
maxSpareThreads="75" maxThreads="150" minSpareThreads="5" port="16020"
protocol="HTTP/1.1" redirectPort="8443"/>

Save this change, then restart the Web Publishing Engine. Test that the WPE is working by using the URL http://yourServer/fmi/xml/FMPXMLRESULT.xml?-dbnames it should show you an XML list of database names. If it doesn't, you might need to restart the computer. After this comes back up, your syncs should be noticeably faster, especially for syncs with less than 100 changed records or for no-change syncs.

• Flag records instead of deleting them: MirrorSync is very fast at detecting deleted records. However, it's always faster to sync edits and inserts than to scan for deletions, especially in large tables with more than 100,000 records. If you want to optimize performance, it might make more sense to flag records as being deleted instead of actually deleting them. That makes it into an edit operation, which is not affected by the number of records in the table.

• If you are starting off with an empty clone, and the initial sync is taking a very long time, you can pre-populate the empty clone table. See the next question: "Can I pre-populate my spoke database to speed up initial sync?"

Performance for syncing container data

MirrorSync has two methods for syncing container data.

Method 1: BASE64 encoding to turn binary container data into text. This text is then sent along with other record data using an HTTP POST. Benefits of this method:

• No connection to port 5003 needed, so better compatibility with firewalls
• Doesn't need to establish a connection to FileMaker Server, which speeds up the overall sync
• Simpler process, less 'moving parts' and potential for errors in the sync
• Does not use any concurrent licensing slots on FileMaker Server during the sync
• Will work correctly even if the address of FileMaker Server changes (after the script steps are copied)
• Generally more efficient for smaller container data

Method 2: Connect as a guest of FileMaker Server and transfer the data using FileMaker's binary protocol on port 5003. Non-container data is sent using an HTTP POST, and then MirrorSync combines the container and non-container data in a separate step. Benefits of this method:

• Uses 37% less network bandwidth, so it's often more efficient for large amounts of container data
• Less likely to run out of memory or crash FileMaker Go running on an iOS device (BASE64 encoding can be CPU intensive and requires lots of RAM)
• Generally more efficient for large container data

Which method is used depends on the $$base64Limit global variable in the MirrorSync script. This is set to 10,000,000 by default (10 million). Any container data smaller than this is sent using method 1, and anything larger than this limit is sent using method 2. You can set this global variable higher or lower in the MirrorSync customization script to control which method you prefer for various file sizes. Setting it to 0 will always use method 2.

Can I pre-populate my spoke database to speed up initial sync?

Yes. There are several approaches to this:

• If you want the spoke user to get a complete copy of the file from the server, and then just delete certain records based on filtering criteria, then you do not need to do anything special. Just select the option in the spoke database to 'Sync with an empty clone or complete copy of the main file' when you select the spoke database in the MirrorSync configuration. When you download the database or create a download link, specify the option for a full copy. The initial sync will run very quickly in this configuration, and will take care of deleting records that the user does not have access to based on filtering criteria.

• If you're syncing with a copy of the server file, but for file size reasons, you don't want them to download a complete copy of the database, follow these steps:

  1. Duplicate your main database and delete all the records that you don't want the spoke users to download
  2. Host this duplicate file on the same FileMaker Server as the main file.
  3. In the configuration editor, select the option to 'Sync with a separate mobile file', and then select that duplicate file as the mobile file. With these steps finished, when your users use the download link feature of MirrorSync, it will download the trimmed-down duplicate file instead of a copy of the main file.

• If you need to selectively delete certain records prior to them downloading a copy of the file, but you need to customize this for each user:

  • This is pretty simple - just make a spoke copy for that user, delete whatever data you don't want the user to have, and then send it to the user. Keep in mind that if the user has record-level access to all the data, and if you haven't set up any filters in your MirrorSync customization script, then the initial sync will put those deleted records right back into the spoke file, so make sure you only delete the records that the user won't get during the sync.

• If you are syncing with a separate mobile file that you want to pre-populate, here are the steps to follow:

  1. Import the records you want from the main file into the mobile file. Be sure to uncheck the box that says "Perform auto-enter options while importing". The primary keys, modification timestamps, and creation timestamps need to match between the mobile file and the server file. Make sure that you import most or all the records that the user will have access to during the sync - it's faster for MirrorSync to sync an empty table than a table that has a small fraction of the desired records.
     • If you do not import all the records in a table, be sure to modify your MirrorSync customization script (if using XML) or your SQL qualifier (if using JDBC) to exclude the records that you do not import. If you fail to do this correctly, it will lead to records on the server being deleted during the initial sync (see more details below).
  2. After the import, open your separate hosted mobile solution. Go to the MirrorSync layout and click the internal tab, find the record with type 'Server' (create new record if necessary and set type to 'Server'). Modify the sync4 field (you'll need to switch to layout mode to see which one that is), and set it to the timestamp when you ran the import, using the format 'YYYY-MM-DD hh:mm:ss' (in 24 hour time). Note: If you are not syncing with FileMaker Server (ex. MySQL<->FileMakerPro/Go) you will make this change in the spoke file.
  3. Make sure that your spoke users run a sync before making any database changes. Any new records that they create in the mobile file will be deleted during the initial sync. It's OK for them to add new records normally after the initial sync has successfully completed.
  • It's easy to make a mistake in this process. Don't hesitate to contact 360Works support if you'd like help with this.

If you do an initial sync with a pre-populated database, and it causes records to be deleted on the server, it's because the records being deleted were 1) not imported into the spoke files, and 2) were not excluded by the MirrorSync customization script. In this case, MirrorSync sees that these records are missing from the spoke file, and also that they exist on the server with an old modification timestamp, so it assumes that they were deleted on the spoke file and proceeds to delete them on the server.

You can test to see if the customization script is excluding the records (as it should) by going to the sync layout in question, finding all records (or a specific one that you want to test), then running the MirrorSync customization script with no parameters. If the record is removed from the found set, then it is being correctly filtered out. If it remains in the found set, then it is not being filtered out, and will be deleted on the server during the initial sync if that record does not exist on the spoke file.

Security

How do I enable SSL encryption for syncing?

See the 'Can MirrorSync send encrypted data? What about SSL enabled FMS?' documentation on this topic.

How do I make all of my users log in with the same username and password to the hub database?

In the 'spoke' section of the MirrorSync configuration process, select the option to log in with 'Hub database credentials configured in previous screen'. This will use whatever username and password you use for configuring MirrorSync as the username and password for running the actual syncs, regardless of the username / password used to open the FileMaker Go/Pro database. If you select this option, users will not be prompted for a password when they run the sync. For FileMaker Server, you may filter the records as described in the 'How can I filter records on a per-user basis if everybody is logging in with the same FileMaker account?' section below. For non-FileMaker databases such as SQL Server, you can filter records as described in 'How Can I Filter Records Using SQL?' section below.

If you select the option to sync with the hub database credentials, then whatever username and password you enter during the MirrorSync configuration for the hub will also be used for users to sync with it. This is convenient, as users will not be prompted to enter a password, but it is potentially less secure, since all users will be treated as the same user by the hub databases. It also makes it possible for a malicious user who is able to open the FileMaker client database to sync with the server, without knowing any passwords, so be sure to put a strong password on the client file to prevent this. The hub password is not embedded or stored in the client file, it is only stored on the MirrorSync server.

The default option, which is to use FileMaker client credentials, is a more secure option. The user will be prompted for their password when they first sync. The sync will then use their username and password for all requests to the hub database server. This then makes it easy to filter records on a per-user basis, and the user's account name will be used for any actions taken on the FileMaker Server.

Customizing MirrorSync

What is the MirrorSync customization script?

Many of the following tips reference the MirrorSync customization script. This is a portion of the main MirrorSync script (named "MirrorSync 6([yourConfigName])") where you can add custom behavior before, during, and after the synchronization process. While previously the MirrorSync Customization script was isolated into its own script, it has since been merged into the main script as described previously. The MirrorSync customization script and its documentation start on line 138 of the main script with the comment #CUSTOMIZATION INSTRUCTIONS.

The MirrorSync customization script is ONLY used with FileMaker client, or with FileMaker Server when the .XML option is selected. When using JDBC, the only server-side customization that can be done is filtering which records are synced, and that is done using an SQL statement instead of the MirrorSync customization script.

Note:

If you use the MirrorSync customization script to filter records, MirrorSync will delete the excluded records from the spoke (client) database if they are already present. This can present a problem if there are cascading deletions that will trigger to delete other related records, because MirrorSync won't be "expecting" those records to be deleted in the spoke. If this occurs, the cascading deletions will cause those same related records to be deleted on the hub.

To resolve this without simply disabling the cascading deletions, you just need to use the MirrorSync customization script to exclude those related records from the sync.

I don't want to sync my entire database to my spoke users. Can I just sync certain tables, fields, or records?

That's a three-part question with a three-part answer.

Syncing certain tables is very easy: When you're configuring the sync using the MirrorSync configuration utility, only include layouts corresponding to the tables that you want to sync.

Syncing certain fields is also very easy: Just don't include any fields on your sync layouts except the ones that you want to sync.

Syncing certain records is only slightly less easy: For SQL databases, or for FileMaker Server with the JDBC option, jump to the 'How Can I Filter Records Using SQL?' topic below. For FileMaker Server with XML, edit the 'MirrorSync Customization' script. Go to the 'hub' section (not the 'spoke', that should almost never be modified) and follow the instructions in the 'FindChanges' documentation there to constrain the found set to just the records you want to sync for the current user. Remember that the script will be running as the user doing the sync, so you can use the Get( AccountName ) function to determine which records are available. Be careful regarding the use of $$globalVariables - they will NOT be persistent across multiple calls to the customization script (except for $$MIRRORSYNC_USERTOKEN - see next question).

Any records that are excluded by your search criteria will not be synced to the user. In addition, if those records were previously synced, they will be deleted from the user's device. This makes it very easy to create check-in / check-out workflows with MirrorSync, where jobs are synced to iPads, completed, and then deleted from the device.

To test your customization script, go to the sync layout that you want to test, show all records, and run the MirrorSync customization script. If you don't get the results you expect, use the Script Debugger to tweak it.

Another approach to limiting record access is with FileMaker's record level access restrictions. Any restrictions you place on the user's privilege set will be respected by MirrorSync, and the user will only receive those records when they do a sync. Records which were previously visible which switch to non-visible will be deleted on the next sync. However, we don't recommend this approach for large record sets, because it is much, much slower than the customization process described above.

If you decide to limit the synced records, you may choose to distribute an empty clone to your users instead of a full copy of the database. That way, the initial sync will only pull the records into their spoke file that they have access to on the database. Keep in mind that if they have access to many thousands of records, that could make the initial sync slower than if you distributed a full copy of the database that already contains all the record data, and let the initial sync delete the extra records.

How can I filter records on a per-user basis if everybody is logging in with the same FileMaker account?

Use the $$MIRRORSYNC_USERTOKEN to accomplish this. This is a special global variable that MirrorSync passes from the client to the server when a sync runs. You can set this global variable to whatever you want on the client, and it will be set on the server so that the MirrorSync customization script can use it when searching for records. This global variable is the one exception to the 'no global variables' rule for server-side filtering, so use it any way you want. You'll notice that it is nested in a conditional in the MirrorSync customization script that sets it. That 'If' statement prevents it from being set on the server, which is important because that would override the value sent by the client.

How can I filter records using SQL?

Yes, if your hub is a SQL Server, Oracle, MySQL, or similar database. In the screen where you select your primary keys, check the box titled 'Custom SQL qualifier'. This will pop open a dialog where you can write any arbitrary SQL qualifier to restrict which records the user will get. Note that you can use the {MIRRORSYNC_USERNAME} and {MIRRORSYNC_USERTOKEN} (see previous tip) substitution variable to customize the search based on the account name that they used to log into FileMaker Go/Pro with - even if you are using a single shared account for logging into the hub database.

Keep in mind that {MIRRORSYNC_USERNAME} will be in whatever upper / lower case the user entered it, and also remember that FileMaker SQL queries are case-sensitive by default. This means that if you were expecting the username to be 'bsmith', but the user logged in with 'BSmith', then {MIRRORSYNC_USERNAME} will contain 'BSmith'. If this is a concern, it's a good idea to use the UPPER function in the SQL qualifier to make the username and the field being searched against both upper case.

You can test your SQL qualifier in this screen to make sure that there are no syntax errors. However, this will only work when editing a configuration, not when creating a new one, so complete the configuration without filtering and then edit it to test the SQL filtering.

MirrorSync will append your SQL qualifier to the auto-generated SQL ending with the table name. For instance, if you used the SQL qualifier "WHERE (category='active')", then when MirrorSync generates the complete SQL, it would look like this:

SELECT "column1", "column2" FROM "Your Table Name" WHERE (category='active') AND modificationTimestamp={TS '2016-07-15 9:00:00'}

It's a good idea to put your search terms in parentheses, because MirrorSync will append an AND condition with the timestamp. For instance, if you used this search qualifier: "WHERE status='active' OR status='pending'", that will be re-written as "WHERE status='active' OR status='pending' AND modificationTimestamp={TS '2016-07-15 9:00:00'}", which would give incorrect results. However, if you wrote your query as "WHERE (status='active' OR status='pending')", then it would be rewritten as "WHERE (status='active' OR status='pending') AND modificationTimestamp={TS '2016-07-15 9:00:00'}", which would give the desired results.

• If you want to write qualifiers that use related field data, you can do that using join statements, like this. Note that you can use table aliases for the additional tables in the query, but NOT the base table:

INNER JOIN secondTable T2 ON ("YourTableName".PK=T2.FK) WHERE ("Your Table Name".category='active' AND T2.someOtherFlag=1)

This would then only retrieve items from the base table where the base table has the category set to 'active' and the second related table has someOtherFlag=1.

• If you want to filter records based on the current user's account name, you can use the {MIRRORSYNC_USERNAME} substitution, like this:

WHERE assignedAccountRep = '{MIRRORSYNC_USERNAME}'

If you are using FileMaker Server as the hub database, there are a few tips to be aware of. FileMaker's JDBC driver uses mostly standard SQL syntax for writing queries. For reference, you can view Claris's SQL Reference documentation. Here are a few FileMaker-specific tips to be aware of:

• If using date vales in your qualifier, use the date format DATE 'yyyy-mm-dd'. For instance, if you wanted to only sync records since February 1st, 2016, you would use this custom SQL qualifier:

WHERE myDateField >= DATE '2016-02-01'

• If you want to filter records based on some user selection, then set the $$MIRRORSYNC_USERTOKEN global variable on the client before running the sync. This will then be passed to the MirrorSync server, where it is available using the {MIRRORSYNC_USERTOKEN} substitution. For instance, you could pop up a custom dialog asking the user for a job number. Their selection would be stored to the $$MIRRORSYNC_USERTOKEN global variable, and could then be filtered on the server by using this custom SQL qualifier:

WHERE jobNumber = '{MIRRORSYNC_USERTOKEN}'

• You can combine all these qualifiers using AND statements. For instance, let's say that you only wanted to sync active jobs, where the related user in the staffMembers table matches the current username, due in either 30, 60, or 90 days, depending on a user selection. The SQL qualifier would look like this:

INNER JOIN staffMembers T2 ON ("Your Table Name".userId = T2.primaryKey) WHERE T2.accountName = '{MIRRORSYNC_USERNAME}' AND "Your Table Name".jobStatus = 'active' AND "Your Table Name".dueDate <= (CURRENT_DATE + {MIRRORSYNC_USERTOKEN})

• FileMaker's JDBC driver is very fast when searching on a single table, but it can be slow when doing joins. If you have a large set of records and find that a custom SQL qualifier for related fields is making the sync take a very long time, it might be worthwhile to switch to XML and compare sync speeds.

Refer to the FileMaker SQL Reference PDF for more details. If you would like our help writing or reviewing a custom SQL qualifier, that is outside the scope of our free support, but it is a service that is available at our standard hourly rate.

How can I tell if a record created in FileMaker Go has ever been synced with the server?

During configuration, select any field as a write-back value (see write-back values in the primary key section, above). Make sure that value is only set to auto-enter a value on the server, by checking that Get( MultiuserState ) = 2`. If the field is blank, the record has not yet been synchronized. If the field contains a value, then it was either created on the server, or created in FileMaker Go and synced with the server.

See the following example with a field named "tag_synced":

Tag_synced Field Definition and Calc

Defining tag_synced as a writeback field

Another approach you can take is by modifying the didInsert section of the customization script (see next question), but the write-back approach is significantly faster when syncing.

How can I run a script on the server when a record is synchronized?

Maybe you want to send an e-mail to an administrator when a user completes a delivery, creates a request for a quote, or deletes a job in production. To do these types of operations, modify the MirrorSync customization script. There is a section in the script for 'DidUpdate', 'DidInsert', and 'WillDelete'. Make sure you modify those in the 'Hub' section - it's very rare that these should be modified on the spoke. In the relevant section of the script, check the current layout, and if it's the one where you want to take action, go ahead and do whatever you want in this section. You can freely go to other layouts, do searches / inserts / updates / deletions, send e-mails, or take any other action, as long as you finish the script on the same layout and record where you started.

If MirrorSync detects that the record was modified as a result of a 'DidInsert' or 'DidUpdate' script, it will re-do the client search, taking into account any scripted record restrictions in the 'FindChanges' section. For example, if you want jobs to be removed from the user's iPad when a record on the 'SyncJob' layout has its status changed to 'Completed', take these steps:

1. Modify the 'FindChanges' to exclude records whose status is 'Completed' when Get( LayoutName ) = SyncJob. (To do this type of search in FileMaker, enter Find mode, Set the status field to 'Completed', Omit the current record/request, and then Constrain Found Set)
2. Modify the 'DidInsert' section to blank out the modification timestamp when Get( LayoutName ) = SyncJob.
3. Do the same thing in the 'DidUpdate' section.

Now, whenever a spoke user updates or inserts a record in the 'SyncJob' table, the script will blank out the modification timestamp. This triggers a change, which causes MirrorSync to re-do the search, which will exclude the complete job from the search, which will automatically remove it from the iPad.

You might ask, why blank out the modification timestamp? The answer is that it actually doesn't matter what you do, as long as you make some change to the record at all. Blanking out the modification timestamp is a harmless change, because FileMaker will immediately replace the blank value with a new timestamp. You could just as well set any field on the record to its current value, or make some more meaningful change, such as filling in a date field with the data that the job was completed.

I don't want users to see any dialogs when the sync runs, can I prevent them from showing?

Yes, set the global variable $$SILENT_MODE to "1" in the MirrorSync customization script. This will prevent MirrorSync from showing any dialogs unless an error occurs, and for required information like login and conflict resolution.

• If you would like to also skip the login dialog, you can select the spoke option to sync with the hub database credentials. This will prevent MirrorSync from prompting the user to enter their password.
• If you would like to prevent the conflict resolution dialog from displaying, pick any option other than 'user decides' in the conflict resolution section of the MirrorSync configuration process.

I don't want users to see the MirrorSync window at all when a sync runs. Can I prevent that from showing?

If you are running MirrorSync on FileMaker Pro, then yes. Set the $$MIRRORSYNC_HIDE_WINDOW variable to 1 in the MirrorSync customization script. This will move the MirrorSync window off-screen, where the user cannot see it. This is not recommended for large sync operations that will take longer than a few seconds, because without that status display, users may think that their computer is frozen.

If you are running MirrorSync on iOS, then this cannot be hidden. That is because MirrorSync needs to open a new window to manipulate the found set without losing the current selection, and new windows cannot be hidden offscreen on iOS devices. The $$MIRRORSYNC_HIDE_WINDOW variable will be ignored on iOS.

Can I block other MirrorSync dialogs from being displayed to the user, or change the wording?

Yes. Using the MirrorSync customization script, all dialogs, even error messages, can be suppressed or modified. Look towards the bottom of the MirrorSync customization script, and you will see a section that includes a dialog for each type of message that is displayed to users during a normal sync. You may freely customize the message displayed to the user, or the title of the buttons (but be sure that the first button still has the same meaning, and the second button still has the same meaning). If you want to hide the dialog from showing, you can disable the script step that displays it, but you must decide which option you would like to select, and exit the script with that button number. You can use this technique to display messages to your users in their preferred language.

Can MirrorSync run automatically when my users finishes a job/invoice/widget? Can it run at startup?

Yes, absolutely. First of all, we recommend setting the 'MirrorSync setup' script as your startup script, or calling it from the startup script. This will check to see if the file is running offline, and whether it has ever been synced before. If not, it will prompt the user to do the initial sync. In addition, the MirrorSync setup script sets the $$MIRRORSYNC_DATABASEID global variable, which is necessary for some primary key strategies see the Primary Key section above.

Remember that the MirrorSync script is a regular FileMaker script that can be set to run at any time by the developer. For example, if there is a critical section of your application that requires users to see what the latest value is before they make a change, trigger the sync when the user navigates to that screen, and again when they are finished editing in that screen. You could make it so that checking a box, clicking a button, or any other condition that you choose can trigger the sync to happen.

Also keep in mind that you can create multiple sync configurations - so you could sync just a certain table, maybe with a different sync direction than usual, and attach that configuration to the button for the fastest possible performance.

Can MirrorSync run automatically every x seconds/minutes?

Yes, you can use an On Timer script to trigger this. However, keep in mind that when the script runs, it commits the current record and pops open a new window that could be open for a few seconds or longer. This could get annoying if it happens in the middle of something that the user is working on. For this reason, we recommend the approach in the previous question of linking your script sync operations to some user-initiated action. The exception to this is unattended computers - if you want to have a computer that is running MirrorSync all the time for purposes of having an extra copy of the database replicated, without a user working on it, then by all means go ahead and use an On Timer script step (and be sure to see the FAQ above about disabling dialogs).

Customizing the MirrorSync layout

Feel free to make cosmetic changes to the 'MirrorSync' layout that is pasted into your solution. However, follow these tips before making changes:

• Make sure you are able to successfully sync on FileMaker Pro and FileMaker Go before making customizations.
• Make a backup copy of the unmodified MirrorSync layout first.
• It is important that you preserve the tabs on the layout, and that the same fields remain in those tabs. The other thing to be careful about is that there are two 1x1 pixel web viewers on the MirrorSync layout. Make sure that those remain visible on the layout.
• If the sync stops working after you make the changes, then you may have inadvertently lost the web viewers on the layout. Restore them from the backup copy of the layout.

Getting rid of duplicates

Let's say that you have two users that independently create new records for the same client. On your server, you want to automatically delete any records that get inserted with the same client name as a record that already exists. The place to put the de-duping script would be the 'DidInsert' section of the MirrorSync customization script. Your script should get the client name (or whatever field(s) you want to use to de-dupe) of the newly inserted record and do a search for it. If only one record comes up, then there is nothing else to do. If multiple records come up, then delete whichever ones you want to get rid of. That will automatically trigger a re-sync to happen after the sync finishes. You could of course do more sophisticated operations such as combining values from the second record into the first, or simply mark the records as potential duplicates for an administrator to clean up later rather than actually deleting them.

Can I build my own front-end for the download link URL feature?

Yes, you can trigger the download link URL from anywhere in order to get a copy of the current file on the server. If you want to do this from FileMaker itself (with the 'Insert from URL' command), or anything else that is not a browser, you will want to disable the JavaScript progress pages that inform the user that the download is being generated. To do this, add a '?status=0' onto the end of the URL, like this:

https://yourServerAddress/MirrorSync/dbDownload/verylongdownloaduuid?status=0

There is an excellent article by Stephen Dolenski, posted on FMForums, that shows how you can notify users when a new version is available. You can read that here.

What should I be aware for very large database synchronization?

For very large databases, here are a few tips:

• In versions of MirrorSync prior to 3.17, there was a limit of 64 gigabytes for the internal sync database. This was increased to 256 gigabytes in version 3.17, and can be increased even high by modifying the 'maxSyncDatabaseSize' in the MirrorSync.xml configuration file. This setting will only apply to NEW sync databases, so changing it will have no effect until your reset the sync data.
• Although MirrorSync can use serial numbers or UUIDs for primary keys, we recommend UUIDs for very large databases. If anything becomes damaged in the internal sync database, it can be almost impossible to bring a very large databases back into sync with serial number primary keys.
• Conflict merging can become slow when the databases are very large. If you do not need to merge field changes, then disable field-level merging in the MirrorSync configuration.
• The bigger the database gets, the more speed benefit you will get by switching from spinning disk hard drives to SSD storage.

MirrorSync.xml Configuration File

The MirrorSync.xml file is a configuration file intended to give users a wide range of customization options for how this instance of MirrorSync will run for all configurations. These options are not available in the MirrorSync configuration utility, and are intended for advanced users. The effects of the parameters within this file are documented within that same file. If you have any confusion about how a parameter will function, please contact support@360works.com before attempting its use, unless another part of this documentation recommends its use.

Note:

Tomcat and MirrorSync will restart after modifying this file, so be sure not to change this file while during an active sync.

Location:

Windows: C:\Program Files\360Works\Applications\conf\Catalina\localhost\MirrorSync.xml
Mac: /Library/360Works/Applications/conf/Catalina/localhost/MirrorSync.xml

This file's location in a Linux installation will vary depending on your installation.

Note for Hosting Provider Installation Users:

The name of this file may vary if you chose "Hosting Provider Installation" and selected a different name for the app than "MirrorSync". If you're unsure of the name of the file, you can find it by looking in the conf/Catalina/localhost directory; the name of the file should match the name that was assigned to the instance on installation. If there are multiple instances of MirrorSync installed on the same server, each will have their own .xml file in this location.

Licensing

How does licensing and pricing work for MirrorSync?

MirrorSync is completely free to use, with no limitations on features, for syncing FileMaker Server with a single copy of FileMaker Pro or FileMaker Go. You can optionally purchase additional server configurations or devices.

The supported configuration types are:

• FileMaker Server with FileMaker clients (Go or Pro)
• FileMaker Server with FileMaker Server
• FileMaker Server with any SQL database (Oracle, MySQL, MS SQL Server, Salesforce, and Amazon RedShift are fully supported. You may also sync with any database with a JDBC driver).
• SQL database with SQL database
• SQL database with FileMaker clients (Go or Pro)

Device (Pro/Go Client) prices range from $95 for one device to $40 each for 100+ devices, depending on the total quantity including the devices about to be purchased, as well as the free device that comes with MirrorSync. The pricing tiers are at 1, 5, 10, 40, 60, and 100.

Please note that the devices on a license are not linked to a single solution on your license, and can be distributed amongst all license solutions.

Go to our online store to see all pricing details.

Is MirrorSync included in the 360Works Portfolio Bundle?

Yes. MirrorSync 3 is free for one device, but Portfolio Bundle subscribers will receive an additional 10 device pack (for a total of 11 devices). Server-to-server configurations are not included in the Portfolio License, and must be purchased separately. There is also a special licensing program for FileMaker Business Alliance (FBA) members; contact us at support@360works.com for details.

Is there a vertical market license type for MirrorSync?

No, but we'll be happy to explain how the existing licensing pricing works for vertical market solutions. Contact 360Works support for details.

Does MirrorSync use any concurrent connections on FileMaker Server?

MirrorSync 2.1 and later does not use any concurrent connections for non-container data (text, number, date, time, timestamps). If you are syncing container fields, then MirrorSync may need to connect to FileMaker Server for the duration of the sync (it disconnects as soon as the sync finishes), depending on what version of FileMaker Pro/Go you are using, as well as the size of the container field contents.

The table below shows an 'X' for any situation where MirrorSync needs to connect as a guest of FileMaker Server.

FileMaker Pro/Go version	Download containers from server	Upload container to server (<10 megabytes)	Upload container to server (>10 megabytes)
11	X	X	X
12	-	X	X
13 and later	-	-	X

Miscellaneous questions

Is MirrorSync localized into any non-English languages? Does it work in other languages?

At this time, MirrorSync localized to English, Japanese, and French. If you would like to offer help with translation, please contact support@360works.com to let us know. You should still be able to add MirrorSync to your solution in other languages. See the previous section on customizing dialogs for tips on localizing your solution.

I only have a spoke copy of my file left, and need to upload it back to the server. What do I do?

If necessary, you can re-upload a spoke copy of the file to FileMaker Server and use it with MirrorSync. Be sure to remove the "Client" record in the MirrorSync table, and keep only the "Server" record. To do this, navigate to the MirrorSync layout, switch to the Debugging tab, and perform a find for "Server" in the type field. Select the inverse of the records found by pressing the pie icon in the status bar. Then, delete those other records.

Troubleshooting/Known Issues

Installer Hangs

If the MirrorSync installer hangs during installation, please force quit the installer and then follow the instructions below to recover. This typically occurs when you've had a previous installation go bad for whatever reason and the 360Works service is in a bad state.

Mac

1. Launch the Terminal and run the following commands:
   • cd /Library/LaunchDaemons
   • sudo launchctl unload com.prosc.TomcatDaemon.plist
2. Now download a fresh copy of MirrorSync and rerun the installer.

Windows

1. Open the Services application
   • You can launch it by searching "Services" in the start menu or launching the task manager (Ctrl + Alt + Del) and going to the "Services" tab.
2. Find the "360Works" service
3. Right click it and click "Stop"
4. Now download a fresh copy of MirrorSync and rerun the installer.

Misc. Installer Trouble

If you're running into trouble with the installer, please try doing the following:

• Run as Administrator
• Temporarily disable firewall
• Temporarily disable anti-virus

Resetting sync data

MirrorSync keeps an internal SQL database of all primary keys, modification timestamps, and internal record IDs for every device being synced. Sometimes this internal database can get out of date, especially if primary keys are modified in the database. If you suspect that this is the problem, you can reset this internal data, which essentially starts MirrorSync over again as if it had never synced anything. However, before doing this, be aware that this will cause all existing spoke files to no longer be able to sync. Don't do this if you have users with unsynced changes in their spoke files! If that's not a concern, you can right-click (or control-click on Mac) the name of the configuration in the MirrorSync admin utility, and select 'Reset SyncData'. After doing this, try the sync again with a new spoke file to see if the problem is resolved.

The ProscTomcatDaemon.jar file could not be started.

Error Message: Could not connect to 360Works Tomcat Daemon: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: java.io.EOFException

If you receive this message during installation, and you have Kerio VPN client installed, you will either need to remove Kerio in order to install MirrorSync, or you will need to install MirrorSync manually. The Kerio VPN client, by default, uses the same port that we use to communicate with the ProscTomcatDaemon.

The maximum number of users are currently using this copy of FileMaker Pro

There is a bug in FileMaker Pro and FileMaker Pro Advanced that may occur when pasting the MirrorSync script, or editing the MirrorSync script. This occurs due to the references found in the script, which will reference both the internal and external IP address. FileMaker then opens both of those references, which triggers the licensing server as two copies of FileMaker Pro. This error would occur for any script that contains both references to a single hosted file.

There are two workarounds for this:

• Upgrade your single user license to a volume license. You can contact FileMaker directly to upgrade your license if you have 5 or more copies.
• Paste the script and let FileMaker close due the licensing error. The script will save appropriately, and should work correctly.

If you require assistance on this issue, please let us know.

I keep getting a 102 'Field is missing' error, but I know my Sync layouts have all the right fields, what's wrong with it?

Make sure that the sync layouts are matching in both the spoke and hub copy. If you have repeating fields, make sure the fields show the maximum number of repetitions that are defined in the field definitions. In other words, if a field can repeat 5 times, make sure the repetitions in the inspector show 5.

Sync fails every time with error: Last sync failed: java.sql.SQLException: Error in table <Sync Layout Name>: Parse Error During Update 1: -1

This issue arises if a syncing solution has more than 998 syncing fields in a single table. The limit for the number of variables FileMaker will allow in a Let statement is 1000. During our Server <-> Client transactions, we require two variables for internal use, so the maximum syncing field limit per table is 998.

Repetitions count as a field in this total. For example, if you have 101 fields and your last field has 100 repetitions that'd be a total of 200 out of the 998 limit.

Getting an error 502 during sync or configuration

This is caused by a configuration in IIS (on Windows) or Apache (on Mac OS X) that times out after when waiting for a request. Sometimes, very complex configurations or operations may take longer than this timeout period to complete, causing the web server to send a 502 error and stop the request.

Testing Timeout

You can test whether the timeout setting is the issue by triggering this URL (replace your server address in the URL), which should wait 3 minutes and then return a successful test result. If you get an error instead, then you definitely need to change the timeout settings as detailed in the rest of this section:

https://yourServerAddress/MirrorSync/timeoutTest?duration=180

Updating Timeout on Windows

Click here for instructions for increasing the time before this timeout.

Updating Timeout on Mac OS X FileMaker Server 16+

To increase this timeout on the Apache webserver that's being used by your FileMaker Server, open up this file in a text editor of your choosing: /Library/FileMaker Server/HTTPServer/conf/extra/httpd-proxy-fmiwebd.conf

• Locate this line:
  • ProxyPass /fmi/ http://127.0.0.1:16021/fmi/
• And change it to this:
  • ProxyPass /fmi/ http://127.0.0.1:16021/fmi/ timeout=600
    • timeout=600 denotes a 10-minute timeout. Adjust as necessary, where 1200 is 20 minutes, 3600 is an hour, etc.
• Now restart FileMaker's built in Apache webserver using the following command in Terminal:
  • fmsadmin restart httpserver

If for some reason you get your timeout up to 7200 (2 hours) and still fail in your testing, subsequent timeout updates need to be changed in an additional file.

File: /Library/FileMaker Server/HTTPServer/conf/httpd.conf

• Locate this line:
  • ProxyPass /MirrorSync ajp://127.0.0.1:42423/MirrorSync retry=1 timeout=7200
    • Keep in mind the /MirrorSync portion is dependant upon your installation method. If you installed using a hosting provider install, this portion will be whatever name you set during the installation.
• Update the timeout=7200 designation to whatever you're updating your mod_proxy.conf to.

Finally, restart the httpserver as your would in the normal process.

"You must have a network connection when you use this software for the first time"

If you are running with OS X 10.9 (Mavericks) and if you have Java 6 installed, the Mavericks installer loses the list of trusted SSL certificates. This prevents MirrorSync from being able to communicate with the license server. To fix this problem, install this Java 6 update from Apple.

If you want to verify that this is really the cause of the problem, look at the MirrorSync log file in C:\Program Files\360Works\Applications\logs or /Library/360Works/Applications/logs. If you see the text "the trustAnchors parameter must be non-empty", then this is definitely the problem.

Missing field <fieldName>, it has either been re-named or deleted in the local file

If you have field-level access privileges that prevent all access to certain fields, you will get this error whenever you try to sync. The solution is to set the MirrorSync script to run with full access on the client and server, as described in 'Does MirrorSync require users to sync with full access accounts?'.

Error 201: "Field cannot be modified"

If you have field-level access privileges that prevent the spoke user from modifying certain fields, you will get this error when records are modified on the server. The solution is to set the MirrorSync script to run with full access on the client and server, as described in 'Does MirrorSync require users to sync with full access accounts?`'.

Setup process leaves blank records in some tables

The MirrorSync setup process creates temporary records in tables to automatically detect primary keys, modification timestamps, and creation timestamps. It attempts to delete these temporary records after it finishes. If the temporary records are not being deleted, check to see if they have a cascading delete rule to a table stored in an external file. If so, check to makes sure that your username and password allows access to this external file, and also that the FMXML extended privilege is enabled for that username in the external file.

Server returned a 500 (Internal server) error

This can happen for several reasons, but the two most common are the Web Publishing Engine running out of memory, or a timeout happening in the Web Publishing Engine. It's not obvious which problem is the root cause unless you look at the Apache log file (for FileMaker Server 13, this is in the /Library/FileMaker Server/HTTPServer/logs directory). If you see an entry like this, it means a timeout occurred:

[error] (70007)The timeout specified has expired: proxy: read response failed from 127.0.0.1:16021 (127.0.0.1)

Fixing a timeout is not hard. Use a text editor to modify the file at /Library/FileMaker Server/Admin/admin-helper/WEB-INF/conf/mod_proxy.conf, and find the line that looks like this:

ProxyPass /fmi ajp://127.0.0.1:16021/fmi

Add a timeout=1200 parameter (this sets a 1,200 second / 20 minute timeout), like this:

ProxyPass /fmi ajp://127.0.0.1:16021/fmi timeout=1200

If it's not a timeout problem, it's probably a memory issue in the Web Publishing Engine. This is more likely in FileMaker Server 11 or 12 than 13 or later but is still possible.

For very large record sets, we recommend using JDBC instead of XML Web Publishing. JDBC is much less susceptible to running out of memory. To do this, simply enable xDBC on your FileMaker Server, and make sure that the fmxdbc extended privilege is enabled for your syncing users. MirrorSync will automatically switch to JDBC if these conditions are met. If switching to xDBC is not an option, or if it does not solve the problem, then you will need to increase the memory allocation for the Web Publishing Engine.

To adjust the Web Publishing Engine memory allocation, edit this file:

Mac: /Library/FileMaker Server/Web Publishing/publishing-engine/jwpc-tomcat/bin/catalina.sh
Windows: C:\Program Files\FileMaker\FileMaker Server\Web Publishing\publishing-engine\jwpc-tomcat\bin\catalina.bat

Look for a line that contains the text "-Xmx512M". The 512 is the amount of megabytes to allocate. Change to a higher number, such as 1024 or 2048 (be sure to keep the 'M' after the number), save the file, and then restart the Web Publishing Engine using the FileMaker admin console. Then try the sync again. FileMaker Server 13 comes with this number pre-set to 2048, but you may still need more if you are syncing records that have a large amount of data.

These memory settings won't take effect unless the WPE is restarted, which you can do using the 'fmsadmin restart wpe' command.

OutOfMemoryError

This means that MirrorSync itself ran out of memory. Sometimes you might see the message "Java Heap Space". This can happen when syncing very large batches of records. This error can also manifest as a "GC overhead limit exceeded" error.

If you get this error during an incremental sync (not the initial sync), and if you are not expecting to get a large number of changes during the incremental sync, check your MirrorSync customization script. Make sure that you are not doing a 'Show all records' or 'Perform find' which would lose the found set - only do a 'Constrain records' to reduce the size of the found set.

If you get this error during an initial sync, and you have tables with more than 100,000 records, then you will probably need to increase the MirrorSync memory allocation for the initial sync, especially when syncing the first device.

MirrorSync has a 512 megabyte allocation by default. This is typically sufficient for syncing change batches of around 200,000 records, although that number gets much smaller if the records contain a lot of information. Now divide that number by the number of devices simultaneously syncing - so if you expect up to 4 people to be syncing simultaneously, MirrorSync can probably handle up to 50,000 changes from each user with the default memory allocation. This assumes no container fields; if you are using container fields with FileMaker Go/Pro 13, then those are temporarily stored in memory in MirrorSync, and may potentially cause a memory error even with a single user.

If you need to increase the memory allocation, first make sure that you're running MirrorSync 2.2 or later, and then modify the setenv file. The process is very similar to the WPE instructions, above, but the file is located here:

Mac: /Library/360Works/Applications/bin/setenv.sh
Windows: C:\Program Files\360Works\Applications\bin\setenv.bat

Look at the line that begins:

CATALINA_OPTS="-Dcom.prosc.tomcat=true -Xmx512M

Modify the -Xmx parameter to allocate the desired amount of memory. Below is an example of an allocation of 1 GB.

CATALINA_OPTS="-Dcom.prosc.tomcat=true -Xmx1024M

The new memory settings won't take effect until Tomcat is restarted, which can be done using the 360Works Admin.jar, which you can restart using these instructions

Reading/Writing Incorrect Timestamp Values from MySQL

MirrorSync uses the timezone configured in the MySQL server. Most of the time, this is fine. However, some applications (such as Magento) may ignore the MySQL configured time zone and instead default to GMT/UTC timezones when writing time values. In that case, MirrorSync may not detect changes made in MySQL and/or it may write incorrect time values to other databases. To fix this, modify the MirrorSync.xml configuration file. Find this entry and change the value to "GMT".

<Parameter name="mysqlTimezone" value="" />

MirrorSync configuration client 'hangs'

This is an issue that we have observed on OS X, where you may get the beachball and the configuration client can lock up / freeze / hang when entering text into a dialog. So far, we have been able to trace all these occurrences to one of the following 3rd party extensions:

• Rescue Time
• BetterSnapTool
• BetterTouchTool
• Caffeine
• QuicKeys

If you encounter this issue, check to see if you have any of these applications running, either by looking at the top-right menubar corner or using the Activity Monitor application. Quit them if they are running. If you are experiencing this problem and you do not have one of these applications running, go to your Users & Groups system preference, click the 'Login items' tab, and then use Activity Monitor to quit out of all applications listed as a login item. If that still does not solve the problem, please contact us at support@360works.com and we will help troubleshoot the problem.

Error 10 occurs during sync

This issue typically occurs when a syncing client opens a connection to the MirrorSync server for a container upload, but takes longer than the connectionTimeout property configured in IIS. This can be solved by making a change to the connectionTimeout Property in the IIS Manager application.

1. From Administrative Tools, open the Internet Information Services Manager application.
2. Choose the top level (server name) node in the tree on the left.
3. Choose the "Configuration Editor" option under the management header (from Features view).
4. Expand the "Section" drop down list and expand the system.applicationHost node, then choose "webLimits"
5. The connectionTimeout property is the first property available, and defaults to 2 minutes.
6. Increase the connectionTimeout from 2 minutes to something more appropriate (30 minutes seems to work well).
7. Be sure to apply the change before closing the IIS manager application.

IIS Manager Configuration Editor

Handshake alert: unrecognized_name

This issue can occur when the host name used to access your SSL enabled FileMaker Server does not match the host name resulting from reverse looking up the Server's IP address. For instance if you have an internal DNS rule providing a CNAME pointing to your FileMaker Server which is available externally by a different primary DNS entry.

Typically, the host name used to reach the server should match the servers established host name. If it does not, such as in the example above the mechanism performing this validation may be turned off.

• Access the setenv.bat or setenv.sh file in your 360Works/Applications/bin/ directory, and modify the first CATALINA_OPTS string. add to the end of the string: -Djsse.enableSNIExtension=false making sure to include a space between the new parameter and the last parameter in the string.
• Restart the 360Works Application Server.

Configuring MirrorSync creates gaps in serial numbers

This is known and expected behavior. When MirrorSync auto-detects primary keys, modification timestamps, and creation timestamps, it does so by creating a new test record, checking to see which fields are writeable and which ones have auto-enter options on them, and then deletes that test record. This will result in the next number in the serial number sequence to be used up.

I forgot my MirrorSync admin password

Re-run the MirrorSync installer; as part of that process you can reset the password to whatever you want. Running the installer will not lose or overwrite any configurations.

I'm getting a MySQL error that starts with 'Packet for query is too large'

MySQL has a default limit of 4 megabytes for BLOBs, which are basically MySQL's version of a FileMaker container field. You can increase this limit by adding these lines (or something similar, read MySQL documentation for full details) to your MySQL configuration file:

max_allowed_packet = 1024M innodb_log_file_size = 1024M

Java Security blocks MirrorSync Configuration client from launching

Old versions of MirrorSync will have an expired developer certificate so Java will keep the applet from launching. The certificate will not be updated on these versions so in order to launch the configuration client you must add the origin site to the exception list. Typically this will be the host name or IP address of the machine running MirrorSync. The origin site should be listed on the Java security prompt ex:

Example Security Dialog

Once you know the origin site, open the Java Control panel (Instructions: Mac | Windows) and click on security. Click on "Edit Site List" and then click "add" and enter in your origin site. If you get a warning about using HTTP, click "continue". Once you have added your origin site to the exception list, you should now be able to launch the MirrorSync Configuration client. You may get a security prompt that looks like this:

Example Security Dialog

Make sure to check the box that says "I accept the risk and want to run this application" and then click run. For more details see the Java documentation on this subject.

NullPointerException when copying scripts

If this occurs, and if you are running MirrorSync on a Linux server, make sure that you have the regular version of Java installed, rather than the headless version.

805 File is damaged. Use Recover

This is often resolved by stopping and starting xDBC in the server admin console

Container data appears corrupt, can't be opened, reduced in size

If you use scripts to insert data into your containers, be wary of the options selected for the "insert file" dialogue. If "compress when possible" or "allow user to choose" is selected, it's likely that some of the container data will be compressed when syncing. On the receiving end, FileMaker doesn't seem to know how to handle the inserted file.

Example Insert File Options Dialog

Starting the 360Works Tomcat server does not start the MirrorSync application

This can happen if you previously had MirrorSync 3 or earlier installed. There is a leftover configuration setting that is obsolete now and prevents startup from completing. To fix this issue, edit the MirrorSync.xml configuration file and remove these two lines:

	<!--This is needed to allow custom database types to be loaded at runtime. It needs to be manually customized if you are running MirrorSync with a non-standard name and need custom database types to be loaded-->
	<Loader className="org.apache.catalina.loader.VirtualWebappLoader" virtualClasspath="/Library/360Works/SyncData4_MirrorSync/*.jar;C:\Program Files\360Works\SyncData4\MirrorSync\*.jar;${360directory}/SyncData4_MS/Extensions/*.jar" />

Error from server: user lacks privilege or object not found: SYNCCLIENT in statement [INSERT INTO SyncClient(timestamp, description, device_id, clone) VALUES (?,?,?,?)]

Restarting the MirrorSync application through the 360Work Admin.jar file will resolve this issue.

There is a duplicate ID in node <spoke or hub> in <layout Name> on <device name>. Duplicate ID: <primary key of duplicate>

This error message indicates that there are one or more duplicated primary key values in the specified device and table. For MirrorSync to function, all primary key values must be unique to that table. Duplicates will need to be removed before this device can complete a sync. To find duplicate primary keys in a FileMaker table, use find mode to search the primary key field of that table for ! .

To prevent duplicates in the future, we highly recommend that you 1) on your auto-enter calculation (if you use one), UNCHECK the box that says 'Do not replace existing value of field' (if any), and 2) change the validation on the primary key in the hub file in the specified table to "always" instead of "only during data entry". After doing so, we also highly recommend that you distribute new spoke files.

Example Field Options Dialog

For one-way syncs from spoke to hub, the spoke database cannot start off with records that match existing records on the hub

Imagine that you have the same contact record with the same primary key present on two different spoke databases, both syncing one-way to the hub. Now imagine that device 1 changes the record. Because the sync is one-way, MirrorSync is unable to write this data to device 2, meaning the user of that device will have the wrong data. Furthermore, every time device 2 changes any field on the record, this will cause a conflict that cannot be merged, because device 1 and 2 have no knowledge of each other, and MirrorSync can't write their changes to each other.

To prevent this scenario, MirrorSync does not allow you to have any records present on the spoke device if 1) that table is one-way from spoke to hub, and 2) any record on the spoke also exists on the hub (indicating that the same record could very well exist on other spokes).

If you get this error, you can solve it by either:

• Empty out that spoke table during the initial sync, so it contains no records
• Switch that table to a bidirectional sync

One or more of the spoke databases have records in them. You must select a mode for the initial sync operation.

This is normal when a server-to-server sync has previously run, but the sync data has been reset. If you want the spoke database to be wiped and replaced with the hub database contents, pick the option to 'Delete spoke'. If you're using UUID primary keys, and you want to keep the data in the spoke database, pick the 'Rebuild sync data' option. For rebuilding, MirrorSync scans all the primary keys in both databases to merge them together.

MirrorSync is restoring deleted records on an incremental sync

Sometimes a user may report that they deleted a record and that during their next sync the record was restored to their database by MirrorSync. There are 3 ways this can happen:

• If one user edits a record, and then another deletes the record, the edit will win conflict resolution, causing the deletion to be re-inserted to the database of the user that deleted it. If just a few records are restored intermittently and it's not happening to just a single user, this is likely what's happening. This is not a bug, it's intentional behavior to prevent new data from being deleted. If someone cared enough to edit the record it seems likely that the deletion was unintentional.
• There are two different databases using the same internal identifier while syncing. Usually this is caused by a user copying another user's database and attempting to use that one concurrently with the original. If this is the case, you'll probably see deletion reversions occurring regularly with two different users; the fix is to supply one of those users with a fresh spoke database to sync from.
• The previous sync from this spoke device got unexpectedly interrupted, was unable to reconnect itself and continue, and had no chance to put the spoke, hub, or itself in a normal state to prepare for the next sync. If this happens the next sync with that spoke database will restore all records that were deleted since the last successful sync.

The rest of this section will deal with the last bullet point, as the causes/solutions for the first two are simple and covered within those lines.

Some typical situations that might cause this if they occur during an ongoing sync:

• A network error that severs the connection between MirrorSync and the hub/spoke (lost wi-fi connection, network outage, etc.)
• An unexpected shutdown or stoppage of one of the resources that are critical to the sync (i.e. a user kills the syncing script or exits the db, the server hosting the hub or MirrorSync crashes, the device the spoke db is on shuts down unexpectedly, etc.)

Why is this necessary?

While aborting a sync, MirrorSync is typically able to write data to its own sync data, the hub, and the spoke to allow it to proceed normally during the next sync. However, in the situations when it can't do that the next sync with that spoke will run in "recovery mode", which means that MirrorSync will attempt to restore all possible records that are present within the hub and that spoke, including deleted ones. Because it wasn't able to prepare itself for the next sync prior to whatever failure occurred, it's impossible for MirrorSync to tell whether a record that's only present in one of the databases was inserted on one database or deleted on the other. To prevent itself from deleting good records, MirrorSync will treat any deletions that occurred since the last successful sync as insertions instead; this will manifest as a rollback to the user. However, it's not truly a rollback, as all changes to records should be retained and no new records should be lost.

To minimize the frequency of these issues:

• Make sure your users have stable, strong internet connections while syncing so the risk of a disconnection is small.
• Train your users not to interrupt a sync if possible. If they have to, they should use the "Abort Sync" button instead of exiting FileMaker, stopping the script, or shutting down their device.
• Sync frequently. This will minimize the impact of recovery mode when it's triggered, as MirrorSync will only have to consider the changes since the last sync.
• Ensure that the server(s) and services hosting your hub database and MirrorSync instance are very stable to prevent server failures from contributing to this.
• Flag records as deleted, instead of actually deleting them. Usually you'd do this by creating a field in each table called "deleted" and setting it to true when you'd like to treat those records as deleted. Note that depending on the scale of the solution, modifying all your existing processes to deal with this new paradigm may be quite time-consuming.

MirrorSync is Restoring Deleted Records While Initially Syncing a Full Copy Spoke

Sometimes while a new spoke database is initially syncing, hub records that were very recently deleted are restored to the hub. This can occur due to an interaction between two little-known features of MirrorSync interacting in an unintuitive way. The features:

• During an initial sync MirrorSync must consider any inserted/modified records in the last 25 hours before the time the user downloaded the database to be modifications by the spoke.
• When a record is deleted in one database and modified in another, MirrorSync will restore the deleted record. See below for details about why these things occur. This means that during an initial sync, MirrorSync will restore any records that were recently modified and deleted on the hub after the spoke database was downloaded.

Explanation:

To understand why, consider this situation: A hub FMS hosting server is in Auckland, New Zealand, and a user with a spoke copy of that hub database is in Honolulu, Hawaii. The New Zealand server's timestamps will be 23 hours ahead of the Honolulu user's modification timestamps.

This means that any modifications that the user makes to the database in the first 23 hours after their download will have modification timestamps that are "in the past" when looked at from the perspective of the hub. If MirrorSync doesn't account for this issue, these modifications will never get synced. To deal with this, MirrorSync will consider all modification timestamps from up to 25 hours before the download to be modifications that were made on the spoke copy.

If the record in question hasn't changed on the hub, this is not a problem. MirrorSync will simply see that the two records are the same, won't make any changes, and the sync will proceed as normal.

However, if the record that's considered to be modified was deleted on the hub database after the user downloaded the database, but before the 25-hour limit, something else will occur: the deletion will be reverted and the record will be reinserted to the hub.

This is because while MirrorSync is evaluating record conflicts (situations where two changes to a record do not agree with one another), it always prefers to preserve information. In a situation where one database has deleted a record and another has modified it. MirrorSync assumes that there was a good reason to modify it, that it's possible that the deletion was unintentional or problematic, and will restore that record, even when the deletion was on the hub.

In most situations this is positive, however in this particular situation the modification coming from the spoke may not actually be a new modification, but MirrorSync must assume that it is for the reasons listed above.

Possible Workarounds

• Advise your users to always immediately sync the spoke after it gets downloaded, so there's very little opportunity for relevant deletions to occur before their initial sync completes
• In the same vein as above, setting the sync to be a server-side initial sync will likely eliminate almost all the occurrences of this issue
• Instead of actually deleting records, simply flag records as deleted

My container data isn't getting downloaded

Certain FMS folder settings will keep the FileMaker backup process from including external container data with your backups, which means that your external container data will not be bundled with your MirrorSync client database download (or will not be pushed to the spoke server, in the case of a server-to-server sync).

Review the following screenshot. If you have both the Additional Database Folder (1) and Container Data Folder (2) settings enabled, you'll see the "Back Up Container Data Folder" (3) option. If this is disabled, the external container data will not be included in backup processes. MirrorSync uses the FMS backup feature, so this means that the container data of databases in that additional folder will not be bundled with your MirrorSync spoke/client database download. For MirrorSync to deliver your external container data in your download, you must enable this setting. The only workaround would be to manually retrieve the container data from the server and put it in place with your client database.

java.lang.IllegalArgumentException: Cannot deserialize value of type `java.util.Date`

The full text of the error: java.lang.IllegalArgumentException: Cannot deserialize value of type `java.util.Date` from String "0000-00-00 00:00:00": not a valid representation (error: Failed to parse Date value '0000-00-00 00:00:00'

This error typically manifests when upgrading a FileMaker Server to version 21 or greater without also upgrading MirrorSync. Reinstall MirrorSync with a version greater than 6.6 to fix this.