360Deploy

From 360Works Product Documentation Wiki
(Difference between revisions)
Jump to: navigation, search
(Setup)
 
(19 intermediate revisions by 7 users not shown)
Line 31: Line 31:
  
 
An Enterprise License is included with the 360Works Portfolio License (http://360works.com/portfolio) at no additional charge. Any current Portfolio License holders will be able to immediately start using 360Deploy.
 
An Enterprise License is included with the 360Works Portfolio License (http://360works.com/portfolio) at no additional charge. Any current Portfolio License holders will be able to immediately start using 360Deploy.
 +
 +
 +
An important note:  360Deploys captures the list of databases when you run your first deployment for a license (and after a license reset).  This is assumed to be all the files in your solution.  Following that, you are allowed to deploy a subset of that list of files.  If you have a large list of files in your solution, and you don't want to deploy them all at once, reach out to support@360works.com and we can hard code the list of databases against your license, so you will be able to deploy a subset of that large list of databases without issue.
  
 
==Setup==
 
==Setup==
First, install 360Deploy on both the Production and Development servers by running the installer for the operating system that you are using. If you are using the same server for both production and development files, or if your development databases are not hosted on a server, this only needs to be run once on the Production server. The install is complete and successful when the 360Deploy page is opened in your default browser. Write down the customized name of your instance (if you chose one during installation) for use later during step 3 in the 360Deploy.fmp12 file.
+
View the documentation here on how to get started:
 +
[https://s3.amazonaws.com/static.360works.com/plugins/360Deploy/360Deploy+Instructions.pdf 360Deploy Instructions]
  
Next, open the 360Deploy.fmp12 file. This file does not need to be hosted on a server, although it can be if you prefer. You can complete the set up process on any machine that has FileMaker Pro installed, but for the best speed, either do this on the production server itself or on a computer on the same local network as the production server. The default login credentials are "360deploy" for the username and password. On first login, you will be prompted to change the password for this account. The file will guide you through the set up process:
+
Linux installation instructions are included with the download but you can also view them [[Linux_Web_Application_Installation | here]]
  
* (Optional) Store server log-in credentials for quicker processing. This will skip over password prompts during the import process.
+
== Archive folder ==
* Populating IP addresses and account credentials for both servers.
+
Every time that you run a deployment, 360Deploy will put a copy of the previous production database into a timestamped directory in the FileMaker Server documents directory. This way, if anything goes wrong (due to a problem with the deployment, a programming error in the new version, or any other reason), you can simply place this timestamped archive version back into the live database directory effectively roll back to the both the structure and data that existed prior to running the deployment.  You can find the archive folder at:
* Choosing which files you want to deploy to the production server
+
* Copying and pasting of a script folder and generate scripts to the production layout
+
  
Make sure to click the "Verify" button after steps 3 and 4 to make sure that you have the Production and Development sides configured correctly. '''Note:''' add your custom name (if necessary) to the end of the IP address in a <IP_ADDRESS>/CUSTOM_NAME format
+
Mac:
 +
<pre>/Library/360Works/Applications/MigrationBackups</pre>
  
 +
Windows:
 +
<pre>C:/Program Files/360Works/Applications/MigrationBackups</pre>
  
'''Required database changes'''
+
Keep in mind that this archive directory is never automatically deleted, and can potentially consume a lot of disk space if your solution is large. It's a good idea to manually clean up this directory periodically.
* Databases should be password protected (required by default in FileMaker Server)
+
* Have administrative credentials for both FileMaker servers
+
* If there is a startup script or shutdown script then either disable these script steps by commenting them out or turning off the script trigger, or create a separate account for running the deploy operation and do not run these steps if running with this account. For example:
+
  
 +
Also keep in mind that if you are using containers with external storage, that container data is shared between the old and new database versions - there is no separate copy placed into the archive folder. This speeds up the import and conserves disk space, but it also means that if you do need to roll back the database version, changes that were made to the external container data are not rolled back. Use FileMaker Server's built-in backup feature to guarantee a complete backup including a separate copy of all container data.
  
[[File:Screen_Shot_2017-08-24_at_4.20.00_PM.png]]
+
== Scheduling after-hour imports ==
 +
The script "Upload Files and Start Migration ( $~configuration_id )" is Server Side compatible, and can be used in a Server Side Schedule, which is our recommended way of doing a scheduled deployment. To use this script, we recommend creating a wrapper script, where you can pass in the id of the configuration to the "Upload Files and Start Migration" script.
  
*A script that is set to run on last window close contains a script step “Close File [Current File]" will cause 360Deploy to hang during the rollout. These scripts steps need to be commented out or an if conditional should be added in the script to skip over this script step.
+
This script does make use of the 360Deploy plugin, so you will need to install that server side if you intend to schedule deployments.
  
== Archive folder ==
+
See the example script in the 360Deploy.fmp12 file called "PSOS : 360Store Deployment"
Every time that you run a deployment, 360Deploy will put a copy of the previous production database into a timestamped directory in the FileMaker Server documents directory. This way, if anything goes wrong (due to a problem with the deployment, a programming error in the new version, or any other reason), you can simply place this timestamped archive version back into the live database directory effectively roll back to the both the structure and data that existed prior to running the deployment.
+
  
Keep in mind that this archive directory is never automatically deleted, and can potentially consume a lot of disk space if your solution is large. It's a good idea to manually clean up this directory periodically.
+
To get the id of your configuration, open the data viewer and evaluate the "Configuration::__id" field while you are on the main layout. This will be the value you pass in.
  
Also keep in mind that if you are using containers with external storage, that container data is shared between the old and new database versions - there is no separate copy placed into the archive folder. This speeds up the import and conserves disk space, but it also means that if you do need to roll back the database version, changes that were made to the external container data are not rolled back. Use FileMaker Server's built-in backup feature to guarantee a complete backup including a separate copy of all container data.
+
== External Data Sources Are Different Between Dev and Prod ==
  
== Advanced Options ==
+
In a multi file solution there will typically be External Data Sources. Now if you have a Dev environment with Dev External Data Sources named one way, and a Prod environment with Prod External Data Sources named a different way, this situation requires a little attention before it is 360Deploy ready.  If you did nothing before running a deployment, your Dev file would be deployed into the Prod environment, but would still be looking for the Dev External Data Sources (which will either not be found, or perhaps worse, actually resolve correctly but now you are using the wrong database).
   
+
*You can optionally edit the "360Deploy post-import actions" script in your development file to run actions that you might want to happen after the import such as creating new account privileges from a table of users.
+
*360Deploy also has the option to include a version field for your solution. The value of this field will be passed to your post-import script as a script parameter to conditionally run any post-import operations
+
*360Deploy also has the option to pause the databases so your users can not continue to add/modify data while the deployment is running. They will however have read access to the files. Using this option may cause the deployment to seem to hang while importing records. It is important to note that when using this option if the process seems to hang on importing records, open the FileMaker Server Admin console and resume the database so the import can continue.
+
  
== Performance ==
+
The solution to this problem is to use Dynamic Data Sources.  Dynamic Data Sources allow us to set an External Data Source to a Global Variable, then set that Global Variable during the Startup script. Whatever we set that variable to is what FileMaker will use to resolve the data source. View more documentation about Dynamic Data Sources here: [https://support.claris.com/s/article/filemaker-16---dynamic-data-source?language=en_US Dynamic Data Sources]
For fastest import speed, run the 360Deploy import process from a copy of FileMaker Pro/Advanced either on the production server, or on a computer on the same local network as the production server. It will not cause a significant slowdown to access your development server over a wide area network.
+
  
Using external container storage provides a modest increase in speed for the import. In addition, it will reduce disk space, because the external container data is shared between the old production database and the new version, instead of being duplicated.
+
Dynamic Data Sources allow us to solve this problem of different data sources between Dev and Prod.  First, change your data sources to Dynamic Data Sources.  Use a Global Variable for each of your data sources.  Then in your startup script, create an if/else branch that will determine whether we are running in the Prod or Dev environment, then set the Global Variable to the appropriate path for your data source.  There are two common approaches here:
 +
 
 +
You can use a naming convention that allows you to programmatically determine the right external data source name.
 +
 
 +
So for example, let's say I name my main file "MySolution", I would name the external data sources after it: "MySolution_Invoices" and "MySolution_Inventory". In my startup script, I can determine the proper external data source by doing:
 +
Set Variable: $$invoicesExternalDataSource: Get(FileName) & "_Invoices"
 +
 
 +
Alternatively, and a little bit less clear, is you can take advantage of the fact that 360Deploy preserves the data from the prod file.  So you could store the name of the external data source inside a field in the prod file, and have your startup script reference that field when setting up the dynamic data source. This approach is more flexible, because you could have different files follow different naming conventions for their external data sources, but it's a little harder to follow.
  
== Scheduling after-hour imports ==
 
When you click the deploy button, you'll have a choice to schedule the import for any date and time that you choose. Keep in mind that for this to work, the 360Deploy.fmp12 must remain open and running in FileMaker Pro until the scheduled time. In addition, since you will not be present to click on any dialogs, it's a good idea to re-run the final step of pasting the import script into your solution (step 8).
 
  
 
== Miscellaneous tips ==
 
== Miscellaneous tips ==
 +
 +
=== How are user accounts, value lists, and fonts handled during a deployment? ===
 +
By default accounts, value lists, and fonts will come from the Prod file during a deployment. However this is configurable. On the the layout show below, click on the settings wheel:<br>
 +
<br>
 +
[[File:Deploymentsettings.png|800px]]<br>
 +
<br>
 +
This will open the settings page where you can choose the source for accounts, value lists and fonts:<br>
 +
<br>
 +
[[File:Deploymentsettings2.png|800px]]<br>
 +
<br>
 +
Once you have set the source for these, just close the settings page using the X in the upper <b>left</b> corner.
 +
 
=== Deleting / adding / renaming fields ===
 
=== Deleting / adding / renaming fields ===
Renaming or adding fields in the development database will work fine during the import. Deleting fields will fail, because the import is done by matching order, and so deleting a field on either development or production will cause the remaining fields to 'shift', so the wrong data will be imported into the wrong fields. To solve this problem, either don't delete fields (you can rename them to indicate that they are unused), or delete them from both the production and the development versions - that way the order will still match.
+
Renaming or adding fields in the development database will work fine during the import. If you add fields in your development copy, be aware that this field will be empty after you migrate these changes to the production server, as there will be no data on the production server to populate it.
 +
Deleting fields will work as expected, the field will be removed from the production copy.
 +
 
 +
=== Renaming tables ===
 +
360Deploy uses the FMDataMigration tool, which will match up tables by id and name first, if no match then by name alone, then by id alone.  Renaming tables will work fine as long as a new table is not created using the renamed tables previous name.
  
== Advanced settings ==
+
=== Viewing Errors ===
If you would like a post-import script to run that can set default values for fields, do data parsing from previous versions, etc., this can be accomplished by customizing the '360Deploy post-import actions' script. Your script can receive a parameter with the version number of the production database that is being upgraded. Click the 'Advanced options' and follow the instructions there.
+
If there is an error with a deployment, there will be a record created in the @AppLog table. You will need to open this layout manually and scroll to the last record. Look in the "errorDescription" field for more information about the most recent error.
  
==Troubleshooting==
 
===Import hangs with no dialog on screen===
 
This probably indicates that a startup script is halting the process. Add a section like the screenshot shown above at the top of the startup script to exit the startup script for automated deployments.
 
  
===Issues with Deploy converting comma (',') character to decimals ('.')===
+
<center>[[File:ReturntoMainPage.png| 140 px |link=http://www.360works.com/products/ |alt= 360Works Plugins for FileMaker |FileMaker Help]]</center>
This is likely due to differences in region/language settings on the servers. Make sure the whatever options selected in the deployment environment closely match the options on the production server. On Mac, these can viewed by going to Language & Region, selecting a region, and then going to Advanced to see what the grouping and decimal delimiters are. On Windows, go to Region, Formats, then Additional Settings to ensure the decimal and digit grouping characters are the same as the other server. Note that many regions use a space to serve as a grouping delimiter which is hard to notice unless you click into the text field.
+

Latest revision as of 18:51, 27 July 2023

Contents

[edit] Requirements

[edit] Overview

360Deploy is a product that simplifies the tedious process of deploying changes in a development database to an existing production database. This can be done with the click of a button while eliminating human error and reducing the downtime traditionally associated with the process. All that is required is some simple configuration, and click Deploy!

For a better understanding, consider the typical FileMaker file development scenario of a production and development server. The production server continuously hosts solutions for clients. The development/staging file is the database where alterations are being made by developers to enhance utility and better serve client's needs. Overhauling the old file to reflect the new one typically involves a process during which production is shut down and administrators sift through tables, scripts, and fields to reflect updates. We seek to bypass this arduous task by automating the entire series of events.

The entire process takes only as long as it takes to import data based on your database size, during which the databases are paused. This allows files to be updated without worrying about how often or when the changes occur. The set up consists of installing our plug-in in the FileMaker client running the deploy file, and some simple script copy/pasting.

[edit] Why use 360Deploy?

Altering live databases has traditionally been a tedious, intrusive task. It consists of server downtime and technical labor at the expense of company resources. 360Deploy is capable of doing that in minutes by seamlessly automating the process. What our software does is create a clone of the development database, transfers it to the production server, and then imports data from production into the clone to absorb the new architecture. As a safeguard, a back-up of the production file is also created on the machine before the imports occur should clients wish to revert back to the old structure.

  1. Ideal for frequent upgrades to a databases of any size
  2. Minimizes file downtime, only pausing file temporarily during import of data
  3. Drastically reduces complexity and time required to upgrade new database architecture
  4. Includes import of scripts and calculations
  5. Backs up production file in event changes want to be reverted

[edit] System requirements

  • 360Deploy requires FileMaker Pro or Advanced 16 to run the actual import process.
  • Production databases must be hosted on FileMaker Server 14 or later
  • For technical reasons, FileMaker Cloud is NOT compatible with 360Deploy

[edit] Licensing

There are four types of licenses for 360Deploy. Pricing for each license type is available at http://360works.com/360Deploy by clicking the 'pricing' button.

  • A free demo license, which can deploy to any number of servers. It has one important limitation, which is that it can only be used to deploy databases named '360Deploy Demo Solution'
  • Express Edition: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to a single deployment server
  • Enterprise License: This allows deployment of unlimited FileMaker solutions to a single deployment server. It is not legal to use this license for multiple different clients, for instance in a shared hosting server or vertical market hosted server.
  • Solution Bundle License: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to unlimited deployment servers. The deployed solution can have different names for different customers.

An Enterprise License is included with the 360Works Portfolio License (http://360works.com/portfolio) at no additional charge. Any current Portfolio License holders will be able to immediately start using 360Deploy.


An important note: 360Deploys captures the list of databases when you run your first deployment for a license (and after a license reset). This is assumed to be all the files in your solution. Following that, you are allowed to deploy a subset of that list of files. If you have a large list of files in your solution, and you don't want to deploy them all at once, reach out to support@360works.com and we can hard code the list of databases against your license, so you will be able to deploy a subset of that large list of databases without issue.

[edit] Setup

View the documentation here on how to get started: 360Deploy Instructions

Linux installation instructions are included with the download but you can also view them here

[edit] Archive folder

Every time that you run a deployment, 360Deploy will put a copy of the previous production database into a timestamped directory in the FileMaker Server documents directory. This way, if anything goes wrong (due to a problem with the deployment, a programming error in the new version, or any other reason), you can simply place this timestamped archive version back into the live database directory effectively roll back to the both the structure and data that existed prior to running the deployment. You can find the archive folder at:

Mac:

/Library/360Works/Applications/MigrationBackups

Windows:

C:/Program Files/360Works/Applications/MigrationBackups

Keep in mind that this archive directory is never automatically deleted, and can potentially consume a lot of disk space if your solution is large. It's a good idea to manually clean up this directory periodically.

Also keep in mind that if you are using containers with external storage, that container data is shared between the old and new database versions - there is no separate copy placed into the archive folder. This speeds up the import and conserves disk space, but it also means that if you do need to roll back the database version, changes that were made to the external container data are not rolled back. Use FileMaker Server's built-in backup feature to guarantee a complete backup including a separate copy of all container data.

[edit] Scheduling after-hour imports

The script "Upload Files and Start Migration ( $~configuration_id )" is Server Side compatible, and can be used in a Server Side Schedule, which is our recommended way of doing a scheduled deployment. To use this script, we recommend creating a wrapper script, where you can pass in the id of the configuration to the "Upload Files and Start Migration" script.

This script does make use of the 360Deploy plugin, so you will need to install that server side if you intend to schedule deployments.

See the example script in the 360Deploy.fmp12 file called "PSOS : 360Store Deployment"

To get the id of your configuration, open the data viewer and evaluate the "Configuration::__id" field while you are on the main layout. This will be the value you pass in.

[edit] External Data Sources Are Different Between Dev and Prod

In a multi file solution there will typically be External Data Sources. Now if you have a Dev environment with Dev External Data Sources named one way, and a Prod environment with Prod External Data Sources named a different way, this situation requires a little attention before it is 360Deploy ready. If you did nothing before running a deployment, your Dev file would be deployed into the Prod environment, but would still be looking for the Dev External Data Sources (which will either not be found, or perhaps worse, actually resolve correctly but now you are using the wrong database).

The solution to this problem is to use Dynamic Data Sources. Dynamic Data Sources allow us to set an External Data Source to a Global Variable, then set that Global Variable during the Startup script. Whatever we set that variable to is what FileMaker will use to resolve the data source. View more documentation about Dynamic Data Sources here: Dynamic Data Sources

Dynamic Data Sources allow us to solve this problem of different data sources between Dev and Prod. First, change your data sources to Dynamic Data Sources. Use a Global Variable for each of your data sources. Then in your startup script, create an if/else branch that will determine whether we are running in the Prod or Dev environment, then set the Global Variable to the appropriate path for your data source. There are two common approaches here:

You can use a naming convention that allows you to programmatically determine the right external data source name.

So for example, let's say I name my main file "MySolution", I would name the external data sources after it: "MySolution_Invoices" and "MySolution_Inventory". In my startup script, I can determine the proper external data source by doing: Set Variable: $$invoicesExternalDataSource: Get(FileName) & "_Invoices"

Alternatively, and a little bit less clear, is you can take advantage of the fact that 360Deploy preserves the data from the prod file. So you could store the name of the external data source inside a field in the prod file, and have your startup script reference that field when setting up the dynamic data source. This approach is more flexible, because you could have different files follow different naming conventions for their external data sources, but it's a little harder to follow.


[edit] Miscellaneous tips

[edit] How are user accounts, value lists, and fonts handled during a deployment?

By default accounts, value lists, and fonts will come from the Prod file during a deployment. However this is configurable. On the the layout show below, click on the settings wheel:

Deploymentsettings.png

This will open the settings page where you can choose the source for accounts, value lists and fonts:

Deploymentsettings2.png

Once you have set the source for these, just close the settings page using the X in the upper left corner.

[edit] Deleting / adding / renaming fields

Renaming or adding fields in the development database will work fine during the import. If you add fields in your development copy, be aware that this field will be empty after you migrate these changes to the production server, as there will be no data on the production server to populate it. Deleting fields will work as expected, the field will be removed from the production copy.

[edit] Renaming tables

360Deploy uses the FMDataMigration tool, which will match up tables by id and name first, if no match then by name alone, then by id alone. Renaming tables will work fine as long as a new table is not created using the renamed tables previous name.

[edit] Viewing Errors

If there is an error with a deployment, there will be a record created in the @AppLog table. You will need to open this layout manually and scroll to the last record. Look in the "errorDescription" field for more information about the most recent error.


 360Works Plugins for FileMaker
Personal tools
Namespaces

Variants
Actions
Plug-in Products
Other Products
Navigation
Toolbox