Difference between revisions of "360Deploy"

From 360Works Product Documentation Wiki
Jump to navigation Jump to search
 
(29 intermediate revisions by 7 users not shown)
Line 27: Line 27:
 
* A free demo license, which can deploy to any number of servers. It has one important limitation, which is that it can only be used to deploy databases named '360Deploy Demo Solution'
 
* A free demo license, which can deploy to any number of servers. It has one important limitation, which is that it can only be used to deploy databases named '360Deploy Demo Solution'
 
* Express Edition: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to a single deployment server
 
* Express Edition: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to a single deployment server
* Enterprise License: This allows deployment of unlimited FileMaker solutions to a single deployment server. It is not legal to use this license for multiple different clients, for instance in a shared hosting environment.
+
* Enterprise License: This allows deployment of unlimited FileMaker solutions to a single deployment server. It is not legal to use this license for multiple different clients, for instance in a shared hosting server or vertical market hosted server.
 
* Solution Bundle License: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to unlimited deployment servers. The deployed solution can have different names for different customers.
 
* Solution Bundle License: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to unlimited deployment servers. The deployed solution can have different names for different customers.
  
 
An Enterprise License is included with the 360Works Portfolio License (http://360works.com/portfolio) at no additional charge. Any current Portfolio License holders will be able to immediately start using 360Deploy.
 
An Enterprise License is included with the 360Works Portfolio License (http://360works.com/portfolio) at no additional charge. Any current Portfolio License holders will be able to immediately start using 360Deploy.
 +
 +
 +
An important note:  360Deploys captures the list of databases when you run your first deployment for a license (and after a license reset).  This is assumed to be all the files in your solution.  Following that, you are allowed to deploy a subset of that list of files.  If you have a large list of files in your solution, and you don't want to deploy them all at once, reach out to support@360works.com and we can hard code the list of databases against your license, so you will be able to deploy a subset of that large list of databases without issue.
  
 
==Setup==
 
==Setup==
First, install 360Deploy on both the Production and Development servers by running the installer for the operating system that you are using. If you are using the same server for both production and development files, or if your development databases are not hosted on a server, this only needs to be run once. The install is complete and successful when the 360Deploy page is opened in your default browser.
+
View the documentation here on how to get started:
 +
[https://s3.amazonaws.com/static.360works.com/plugins/360Deploy/360Deploy+Instructions.pdf 360Deploy Instructions]
 +
 
 +
Linux installation instructions are included with the download but you can also view them [[Linux_Web_Application_Installation | here]]
 +
 
 +
== Archive folder ==
 +
Every time that you run a deployment, 360Deploy will put a copy of the previous production database into a timestamped directory in the FileMaker Server documents directory. This way, if anything goes wrong (due to a problem with the deployment, a programming error in the new version, or any other reason), you can simply place this timestamped archive version back into the live database directory effectively roll back to the both the structure and data that existed prior to running the deployment. You can find the archive folder at:
 +
 
 +
Mac:
 +
<pre>/Library/360Works/Applications/MigrationBackups</pre>
 +
 
 +
Windows:
 +
<pre>C:/Program Files/360Works/Applications/MigrationBackups</pre>
 +
 
 +
Keep in mind that this archive directory is never automatically deleted, and can potentially consume a lot of disk space if your solution is large. It's a good idea to manually clean up this directory periodically.
  
Next, open the 360Deploy.fmp12 file. This file does not need to be hosted on a server, although it can be if you prefer. You can complete the set up process on any machine that has FileMaker Pro installed, but for the best speed, either do this on the production server itself or on a computer on the same local network as the production server. The default login credentials are "360Deploy" for the username and password. On first login, you will be prompted to change the password for this account. The file will guide you through the set up process:
+
Also keep in mind that if you are using containers with external storage, that container data is shared between the old and new database versions - there is no separate copy placed into the archive folder. This speeds up the import and conserves disk space, but it also means that if you do need to roll back the database version, changes that were made to the external container data are not rolled back. Use FileMaker Server's built-in backup feature to guarantee a complete backup including a separate copy of all container data.
  
* (Optional) Store server log-in credentials for quicker processing. This will skip over password prompts during the import process.
+
== Scheduling after-hour imports ==
* Populating IP addresses and account credentials for both servers.
+
The script "Upload Files and Start Migration ( $~configuration_id )" is Server Side compatible, and can be used in a Server Side Schedule, which is our recommended way of doing a scheduled deployment. To use this script, we recommend creating a wrapper script, where you can pass in the id of the configuration to the "Upload Files and Start Migration" script.
* Choosing which files you want to deploy to the production server
 
* Copying and pasting of a script folder and generate scripts to the production layout
 
  
Make sure to click the "Verify" button after steps 3 and 4 to make sure that you have the Production and Development sides configured correctly.  
+
This script does make use of the 360Deploy plugin, so you will need to install that server side if you intend to schedule deployments.
  
 +
See the example script in the 360Deploy.fmp12 file called "PSOS : 360Store Deployment"
  
'''Required database changes'''
+
To get the id of your configuration, open the data viewer and evaluate the "Configuration::__id" field while you are on the main layout.  This will be the value you pass in.
* Databases should be password protected (required by default in FileMaker Server)
 
* Have administrative credentials for both FileMaker servers
 
* If the startup script does any write operations (i.e. Set Field or New Record), then either disable these script steps by commenting them out, or create a separate account for running the deploy operation and do not run these steps if running with this account. For example:
 
  
 +
== External Data Sources Are Different Between Dev and Prod ==
  
[[File:Screen_Shot_2017-08-24_at_4.20.00_PM.png]]
+
In a multi file solution there will typically be External Data Sources. Now if you have a Dev environment with Dev External Data Sources named one way, and a Prod environment with Prod External Data Sources named a different way, this situation requires a little attention before it is 360Deploy ready. If you did nothing before running a deployment, your Dev file would be deployed into the Prod environment, but would still be looking for the Dev External Data Sources (which will either not be found, or perhaps worse, actually resolve correctly but now you are using the wrong database).
  
== Archive folder ==
+
The solution to this problem is to use Dynamic Data Sources.  Dynamic Data Sources allow us to set an External Data Source to a Global Variable, then set that Global Variable during the Startup script.  Whatever we set that variable to is what FileMaker will use to resolve the data source.  View more documentation about Dynamic Data Sources here: [https://support.claris.com/s/article/filemaker-16---dynamic-data-source?language=en_US Dynamic Data Sources]
Every time that you run a deployment, 360Deploy will put a copy of the previous production database into a timestamped directory in the FileMaker Server documents directory. This way, if anything goes wrong (due to a problem with the deployment, a programming error in the new version, or any other reason), you can simply place this timestamped archive version back into the live database directory effectively roll back to the both the structure and data that existed prior to running the deployment.
+
 
 +
Dynamic Data Sources allow us to solve this problem of different data sources between Dev and Prod. First, change your data sources to Dynamic Data Sources.  Use a Global Variable for each of your data sources.  Then in your startup script, create an if/else branch that will determine whether we are running in the Prod or Dev environment, then set the Global Variable to the appropriate path for your data source.  There are two common approaches here:
 +
 
 +
You can use a naming convention that allows you to programmatically determine the right external data source name.
 +
 
 +
So for example, let's say I name my main file "MySolution", I would name the external data sources after it: "MySolution_Invoices" and "MySolution_Inventory". In my startup script, I can determine the proper external data source by doing:
 +
Set Variable: $$invoicesExternalDataSource: Get(FileName) & "_Invoices"
  
Keep in mind that this archive directory is never automatically deleted, and can potentially consume a lot of disk space if your solution is large. It's a good idea to manually clean up this directory periodically.
+
Alternatively, and a little bit less clear, is you can take advantage of the fact that 360Deploy preserves the data from the prod file.  So you could store the name of the external data source inside a field in the prod file, and have your startup script reference that field when setting up the dynamic data source. This approach is more flexible, because you could have different files follow different naming conventions for their external data sources, but it's a little harder to follow.
  
Also keep in mind that if you are using containers with external storage, that container data is shared between the old and new database versions - there is no separate copy placed into the archive folder. This speeds up the import and conserves disk space, but it also means that if you do need to roll back the database version, changes that were made to the external container data are not rolled back. Use FileMaker Server's built-in backup feature to guarantee a complete backup including a separate copy of all container data.
 
  
== Advanced Options ==
+
== Miscellaneous tips ==
 
*You can optionally edit the "360Deploy post-impor actions" script in your development file to run actions that you might want to happen after the import such as creating new account privileges from a table of users.
 
*360Deploy also has the option to include a version field for your solution. The value of this field will be passed to your post-import script as a script parameter to conditionally run any post-import operations
 
*360Deploy also has the option to pause the databases so your users can not continue to add/modify data while the deployment is running. They will however have read access to the files. Using this option may cause the deployment to seem to hang while importing records. It is important to note that when using this option if the process seems to hang on importing records, open the FileMaker Server Admin console and resume the database so the import can continue.
 
  
== Performance ==
+
=== How are user accounts, value lists, and fonts handled during a deployment? ===
For fastest import speed, run the 360Deploy import process from a copy of FileMaker Pro/Advanced either on the production server, or on a computer on the same local network as the production server. It will not cause a significant slowdown to access your development server over a wide area network.
+
By default accounts, value lists, and fonts will come from the Prod file during a deployment. However this is configurable. On the the layout show below, click on the settings wheel:<br>
 +
<br>
 +
[[File:Deploymentsettings.png|800px]]<br>
 +
<br>
 +
This will open the settings page where you can choose the source for accounts, value lists and fonts:<br>
 +
<br>
 +
[[File:Deploymentsettings2.png|800px]]<br>
 +
<br>
 +
Once you have set the source for these, just close the settings page using the X in the upper <b>left</b> corner.
  
Using external container storage provides a modest increase in speed for the import. In addition, it will reduce disk space, because the external container data is shared between the old production database and the new version, instead of being duplicated.
+
=== Deleting / adding / renaming fields ===
 +
Renaming or adding fields in the development database will work fine during the import. If you add fields in your development copy, be aware that this field will be empty after you migrate these changes to the production server, as there will be no data on the production server to populate it
 +
Deleting fields will work as expected, the field will be removed from the production copy.
  
== Scheduling after-hour imports ==
+
=== Renaming tables ===
When you click the deploy button, you'll have a choice to schedule the import for any date and time that you choose. Keep in mind that for this to work, the 360Deploy.fmp12 must remain open and running in FileMaker Pro until the scheduled time. In addition, since you will not be present to click on any dialogs, it's a good idea to re-run the final step of pasting the import script into your solution (step 8).
+
360Deploy uses the FMDataMigration tool, which will match up tables by id and name first, if no match then by name alone, then by id alone. Renaming tables will work fine as long as a new table is not created using the renamed tables previous name.
  
== Advanced settings ==
+
=== Viewing Errors ===
If you would like a post-import script to run that can set default values for fields, do data parsing from previous versions, etc., this can be accomplished by customizing the '360Deploy post-import actions' script. Your script can receive a parameter with the version number of the production database that is being upgraded. Click the 'Advanced options' and follow the instructions there.
+
If there is an error with a deployment, there will be a record created in the @AppLog table. You will need to open this layout manually and scroll to the last record. Look in the "errorDescription" field for more information about the most recent error.
  
==Troubleshooting==
 
===Import hangs with no dialog on screen===
 
This probably indicates that a startup script is halting the process. Add a section like the screenshot shown above at the top of the startup script to exit the startup script for automated deployments.
 
  
===Import hangs with import dialog showing===
+
<center>[[File:ReturntoMainPage.png| 140 px |link=http://www.360works.com/products/ |alt= 360Works Plugins for FileMaker |FileMaker Help]]</center>
If this occurs, try resuming the production database using FileMaker Server Admin Console. The production databases are normally paused during the import to prevent users from making changes (which will be lost if the import has already run for that table), but sometimes this can cause the import to hang. In our experience, this is usually a one-time problem: Resuming the database once seems to fix the problem permanently, even for future imports that are not manually resumed.
 

Latest revision as of 18:51, 27 July 2023

Requirements

Overview

360Deploy is a product that simplifies the tedious process of deploying changes in a development database to an existing production database. This can be done with the click of a button while eliminating human error and reducing the downtime traditionally associated with the process. All that is required is some simple configuration, and click Deploy!

For a better understanding, consider the typical FileMaker file development scenario of a production and development server. The production server continuously hosts solutions for clients. The development/staging file is the database where alterations are being made by developers to enhance utility and better serve client's needs. Overhauling the old file to reflect the new one typically involves a process during which production is shut down and administrators sift through tables, scripts, and fields to reflect updates. We seek to bypass this arduous task by automating the entire series of events.

The entire process takes only as long as it takes to import data based on your database size, during which the databases are paused. This allows files to be updated without worrying about how often or when the changes occur. The set up consists of installing our plug-in in the FileMaker client running the deploy file, and some simple script copy/pasting.

Why use 360Deploy?

Altering live databases has traditionally been a tedious, intrusive task. It consists of server downtime and technical labor at the expense of company resources. 360Deploy is capable of doing that in minutes by seamlessly automating the process. What our software does is create a clone of the development database, transfers it to the production server, and then imports data from production into the clone to absorb the new architecture. As a safeguard, a back-up of the production file is also created on the machine before the imports occur should clients wish to revert back to the old structure.

  1. Ideal for frequent upgrades to a databases of any size
  2. Minimizes file downtime, only pausing file temporarily during import of data
  3. Drastically reduces complexity and time required to upgrade new database architecture
  4. Includes import of scripts and calculations
  5. Backs up production file in event changes want to be reverted

System requirements

  • 360Deploy requires FileMaker Pro or Advanced 16 to run the actual import process.
  • Production databases must be hosted on FileMaker Server 14 or later
  • For technical reasons, FileMaker Cloud is NOT compatible with 360Deploy

Licensing

There are four types of licenses for 360Deploy. Pricing for each license type is available at http://360works.com/360Deploy by clicking the 'pricing' button.

  • A free demo license, which can deploy to any number of servers. It has one important limitation, which is that it can only be used to deploy databases named '360Deploy Demo Solution'
  • Express Edition: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to a single deployment server
  • Enterprise License: This allows deployment of unlimited FileMaker solutions to a single deployment server. It is not legal to use this license for multiple different clients, for instance in a shared hosting server or vertical market hosted server.
  • Solution Bundle License: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to unlimited deployment servers. The deployed solution can have different names for different customers.

An Enterprise License is included with the 360Works Portfolio License (http://360works.com/portfolio) at no additional charge. Any current Portfolio License holders will be able to immediately start using 360Deploy.


An important note: 360Deploys captures the list of databases when you run your first deployment for a license (and after a license reset). This is assumed to be all the files in your solution. Following that, you are allowed to deploy a subset of that list of files. If you have a large list of files in your solution, and you don't want to deploy them all at once, reach out to support@360works.com and we can hard code the list of databases against your license, so you will be able to deploy a subset of that large list of databases without issue.

Setup

View the documentation here on how to get started: 360Deploy Instructions

Linux installation instructions are included with the download but you can also view them here

Archive folder

Every time that you run a deployment, 360Deploy will put a copy of the previous production database into a timestamped directory in the FileMaker Server documents directory. This way, if anything goes wrong (due to a problem with the deployment, a programming error in the new version, or any other reason), you can simply place this timestamped archive version back into the live database directory effectively roll back to the both the structure and data that existed prior to running the deployment. You can find the archive folder at:

Mac:

/Library/360Works/Applications/MigrationBackups

Windows:

C:/Program Files/360Works/Applications/MigrationBackups

Keep in mind that this archive directory is never automatically deleted, and can potentially consume a lot of disk space if your solution is large. It's a good idea to manually clean up this directory periodically.

Also keep in mind that if you are using containers with external storage, that container data is shared between the old and new database versions - there is no separate copy placed into the archive folder. This speeds up the import and conserves disk space, but it also means that if you do need to roll back the database version, changes that were made to the external container data are not rolled back. Use FileMaker Server's built-in backup feature to guarantee a complete backup including a separate copy of all container data.

Scheduling after-hour imports

The script "Upload Files and Start Migration ( $~configuration_id )" is Server Side compatible, and can be used in a Server Side Schedule, which is our recommended way of doing a scheduled deployment. To use this script, we recommend creating a wrapper script, where you can pass in the id of the configuration to the "Upload Files and Start Migration" script.

This script does make use of the 360Deploy plugin, so you will need to install that server side if you intend to schedule deployments.

See the example script in the 360Deploy.fmp12 file called "PSOS : 360Store Deployment"

To get the id of your configuration, open the data viewer and evaluate the "Configuration::__id" field while you are on the main layout. This will be the value you pass in.

External Data Sources Are Different Between Dev and Prod

In a multi file solution there will typically be External Data Sources. Now if you have a Dev environment with Dev External Data Sources named one way, and a Prod environment with Prod External Data Sources named a different way, this situation requires a little attention before it is 360Deploy ready. If you did nothing before running a deployment, your Dev file would be deployed into the Prod environment, but would still be looking for the Dev External Data Sources (which will either not be found, or perhaps worse, actually resolve correctly but now you are using the wrong database).

The solution to this problem is to use Dynamic Data Sources. Dynamic Data Sources allow us to set an External Data Source to a Global Variable, then set that Global Variable during the Startup script. Whatever we set that variable to is what FileMaker will use to resolve the data source. View more documentation about Dynamic Data Sources here: Dynamic Data Sources

Dynamic Data Sources allow us to solve this problem of different data sources between Dev and Prod. First, change your data sources to Dynamic Data Sources. Use a Global Variable for each of your data sources. Then in your startup script, create an if/else branch that will determine whether we are running in the Prod or Dev environment, then set the Global Variable to the appropriate path for your data source. There are two common approaches here:

You can use a naming convention that allows you to programmatically determine the right external data source name.

So for example, let's say I name my main file "MySolution", I would name the external data sources after it: "MySolution_Invoices" and "MySolution_Inventory". In my startup script, I can determine the proper external data source by doing: Set Variable: $$invoicesExternalDataSource: Get(FileName) & "_Invoices"

Alternatively, and a little bit less clear, is you can take advantage of the fact that 360Deploy preserves the data from the prod file. So you could store the name of the external data source inside a field in the prod file, and have your startup script reference that field when setting up the dynamic data source. This approach is more flexible, because you could have different files follow different naming conventions for their external data sources, but it's a little harder to follow.


Miscellaneous tips

How are user accounts, value lists, and fonts handled during a deployment?

By default accounts, value lists, and fonts will come from the Prod file during a deployment. However this is configurable. On the the layout show below, click on the settings wheel:

Deploymentsettings.png

This will open the settings page where you can choose the source for accounts, value lists and fonts:

Deploymentsettings2.png

Once you have set the source for these, just close the settings page using the X in the upper left corner.

Deleting / adding / renaming fields

Renaming or adding fields in the development database will work fine during the import. If you add fields in your development copy, be aware that this field will be empty after you migrate these changes to the production server, as there will be no data on the production server to populate it. Deleting fields will work as expected, the field will be removed from the production copy.

Renaming tables

360Deploy uses the FMDataMigration tool, which will match up tables by id and name first, if no match then by name alone, then by id alone. Renaming tables will work fine as long as a new table is not created using the renamed tables previous name.

Viewing Errors

If there is an error with a deployment, there will be a record created in the @AppLog table. You will need to open this layout manually and scroll to the last record. Look in the "errorDescription" field for more information about the most recent error.


360Works Plugins for FileMaker