Difference between revisions of "360Deploy"

From 360Works Product Documentation Wiki
Jump to navigation Jump to search
 
(67 intermediate revisions by 7 users not shown)
Line 1: Line 1:
 +
== Requirements ==
  
 
==Overview==
 
==Overview==
  
360Deploy is a product aiming to simplify the tedious process of deploying a development database onto an existing production database. Developers working on a file can deploy changes made to their database structure at the click of a button while avoiding the downtime and labor traditionally associated with the process. All that is required is the use of our plug-in, some simple configuration, and click Deploy!
+
360Deploy is a product that simplifies the tedious process of deploying changes in a development database to an existing production database. This can be done with the click of a button while eliminating human error and reducing the downtime traditionally associated with the process. All that is required is some simple configuration, and click Deploy!
  
For a better understanding, consider the typical FileMaker file development scenario of a production and development server. The production server is that which continuously hosts solutions for clients, whether through FileMaker clients or via WebDirect. The development/staging file is the one to which alterations are being made by developers to enhance utility and better serve client's needs. Overhauling the old file to reflect the new one typically involves a process during which production is shut down and administrators sift through tables, scripts, and fields to reflect updates. We seek to bypass this arduous task by automating the entire series of events.
+
For a better understanding, consider the typical FileMaker file development scenario of a production and development server. The production server continuously hosts solutions for clients. The development/staging file is the database where alterations are being made by developers to enhance utility and better serve client's needs. Overhauling the old file to reflect the new one typically involves a process during which production is shut down and administrators sift through tables, scripts, and fields to reflect updates. We seek to bypass this arduous task by automating the entire series of events.
  
 
The entire process takes only as long as it takes to import data based on your database size, during which the databases are paused. This allows files to be updated without worrying about how often or when the changes occur. The set up consists of installing our plug-in in the FileMaker client running the deploy file,  and some simple script copy/pasting.
 
The entire process takes only as long as it takes to import data based on your database size, during which the databases are paused. This allows files to be updated without worrying about how often or when the changes occur. The set up consists of installing our plug-in in the FileMaker client running the deploy file,  and some simple script copy/pasting.
  
'''Required database changes'''
+
==Why use 360Deploy?==
* Enable fmapp and fmurlscript for databases in Extended Privileges
+
 
* Databases should be password protected (required by default in FileMaker Server)
+
Altering live databases has traditionally been a tedious, intrusive task. It consists of server downtime and technical labor at the expense of company resources. 360Deploy is capable of doing that in minutes by seamlessly automating the process. What our software does is create a clone of the development database, transfers it to the production server, and then imports data from production into the clone to absorb the new architecture. As a safeguard, a back-up of the production file is also created on the machine before the imports occur should clients wish to revert back to the old structure.
* Have administrative credentials for both FileMaker servers
+
#Ideal for frequent upgrades to a databases of any size
* Comment out script steps in triggers that automatically make any write operations in startup scripts (set fields, new records). I.e.,
+
#Minimizes file downtime, only pausing file temporarily during import of data
 +
#Drastically reduces complexity and time required to upgrade new database architecture
 +
#Includes import of scripts and calculations
 +
#Backs up production file in event changes want to be reverted
 +
 
 +
== System requirements ==
 +
* 360Deploy requires FileMaker Pro or Advanced 16 to run the actual import process.
 +
* Production databases must be hosted on FileMaker Server 14 or later
 +
* For technical reasons, FileMaker Cloud is NOT compatible with 360Deploy
 +
 
 +
== Licensing ==
 +
There are four types of licenses for 360Deploy. Pricing for each license type is available at http://360works.com/360Deploy by clicking the 'pricing' button.
 +
* A free demo license, which can deploy to any number of servers. It has one important limitation, which is that it can only be used to deploy databases named '360Deploy Demo Solution'
 +
* Express Edition: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to a single deployment server
 +
* Enterprise License: This allows deployment of unlimited FileMaker solutions to a single deployment server. It is not legal to use this license for multiple different clients, for instance in a shared hosting server or vertical market hosted server.
 +
* Solution Bundle License: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to unlimited deployment servers. The deployed solution can have different names for different customers.
 +
 
 +
An Enterprise License is included with the 360Works Portfolio License (http://360works.com/portfolio) at no additional charge. Any current Portfolio License holders will be able to immediately start using 360Deploy.
 +
 
 +
 
 +
An important note:  360Deploys captures the list of databases when you run your first deployment for a license (and after a license reset).  This is assumed to be all the files in your solution.  Following that, you are allowed to deploy a subset of that list of files.  If you have a large list of files in your solution, and you don't want to deploy them all at once, reach out to support@360works.com and we can hard code the list of databases against your license, so you will be able to deploy a subset of that large list of databases without issue.
 +
 
 +
==Setup==
 +
View the documentation here on how to get started:
 +
[https://s3.amazonaws.com/static.360works.com/plugins/360Deploy/360Deploy+Instructions.pdf 360Deploy Instructions]
 +
 
 +
Linux installation instructions are included with the download but you can also view them [[Linux_Web_Application_Installation | here]]
 +
 
 +
== Archive folder ==
 +
Every time that you run a deployment, 360Deploy will put a copy of the previous production database into a timestamped directory in the FileMaker Server documents directory. This way, if anything goes wrong (due to a problem with the deployment, a programming error in the new version, or any other reason), you can simply place this timestamped archive version back into the live database directory effectively roll back to the both the structure and data that existed prior to running the deployment. You can find the archive folder at:
  
If [Get ( FoundCount ) = 0]
+
Mac:
 +
<pre>/Library/360Works/Applications/MigrationBackups</pre>
  
New Record/Request
+
Windows:
 +
<pre>C:/Program Files/360Works/Applications/MigrationBackups</pre>
  
End if
+
Keep in mind that this archive directory is never automatically deleted, and can potentially consume a lot of disk space if your solution is large. It's a good idea to manually clean up this directory periodically.
  
[[File:CodeSnippet.png|400px]]
+
Also keep in mind that if you are using containers with external storage, that container data is shared between the old and new database versions - there is no separate copy placed into the archive folder. This speeds up the import and conserves disk space, but it also means that if you do need to roll back the database version, changes that were made to the external container data are not rolled back. Use FileMaker Server's built-in backup feature to guarantee a complete backup including a separate copy of all container data.
[[Link CodeSnippet]]
 
'''Recommended steps'''
 
* If account/password credentials for the Deploy file and both FileMaker Servers match, process will proceed without prompts and pauses.
 
  
==Why use 360Deploy?==
+
== Scheduling after-hour imports ==
 +
The script "Upload Files and Start Migration ( $~configuration_id )" is Server Side compatible, and can be used in a Server Side Schedule, which is our recommended way of doing a scheduled deployment.  To use this script, we recommend creating a wrapper script, where you can pass in the id of the configuration to the "Upload Files and Start Migration" script.
 +
 
 +
This script does make use of the 360Deploy plugin, so you will need to install that server side if you intend to schedule deployments.
 +
 
 +
See the example script in the 360Deploy.fmp12 file called "PSOS : 360Store Deployment"
 +
 
 +
To get the id of your configuration, open the data viewer and evaluate the "Configuration::__id" field while you are on the main layout.  This will be the value you pass in.
 +
 
 +
== External Data Sources Are Different Between Dev and Prod ==
 +
 
 +
In a multi file solution there will typically be External Data Sources.  Now if you have a Dev environment with Dev External Data Sources named one way, and a Prod environment with Prod External Data Sources named a different way, this situation requires a little attention before it is 360Deploy ready.  If you did nothing before running a deployment, your Dev file would be deployed into the Prod environment, but would still be looking for the Dev External Data Sources (which will either not be found, or perhaps worse, actually resolve correctly but now you are using the wrong database).
 +
 
 +
The solution to this problem is to use Dynamic Data Sources.  Dynamic Data Sources allow us to set an External Data Source to a Global Variable, then set that Global Variable during the Startup script.  Whatever we set that variable to is what FileMaker will use to resolve the data source.  View more documentation about Dynamic Data Sources here: [https://support.claris.com/s/article/filemaker-16---dynamic-data-source?language=en_US Dynamic Data Sources]
 +
 
 +
Dynamic Data Sources allow us to solve this problem of different data sources between Dev and Prod.  First, change your data sources to Dynamic Data Sources.  Use a Global Variable for each of your data sources.  Then in your startup script, create an if/else branch that will determine whether we are running in the Prod or Dev environment, then set the Global Variable to the appropriate path for your data source.  There are two common approaches here:
  
Altering live databases has traditionally been a tedious, intrusive task. It consists of server downtime and technical labor at the expense of company resources. 360 Deploy is capable of doing that in minutes by seamlessly automating the process. What our software does is create a clone of the development database, transfers it to the production server, and then imports data from production into the clone to absorb the new architecture. As a safeguard, a back-up of the production file is also created on the machine before the imports occur should clients wish to revert back to the old structure.
+
You can use a naming convention that allows you to programmatically determine the right external data source name.
  
==Technical Description==
+
So for example, let's say I name my main file "MySolution", I would name the external data sources after it: "MySolution_Invoices" and "MySolution_Inventory". In my startup script, I can determine the proper external data source by doing:
 +
Set Variable: $$invoicesExternalDataSource: Get(FileName) & "_Invoices"
  
stuff
+
Alternatively, and a little bit less clear, is you can take advantage of the fact that 360Deploy preserves the data from the prod file.  So you could store the name of the external data source inside a field in the prod file, and have your startup script reference that field when setting up the dynamic data source. This approach is more flexible, because you could have different files follow different naming conventions for their external data sources, but it's a little harder to follow.
  
==Random Notes to help write this wiki==
 
  
3 Choices to deploy DB changes:
+
== Miscellaneous tips ==
1- Make a log of all changes made to development server, then reflect changes onto production server while it's down.
 
2- Separation model - creates compromises, involves workarounds, has trade-offs, requires technical know-how
 
3- Import all data from old data into empty clone of new file, make sure serial numbers are manually adjusted, tedious process
 
  
MirrorSync is an elegant synchronization product that can sync between any combination of FileMaker Pro, FileMaker Go, FileMaker Server, SQL database (MySQL, Oracle, SQL Server, or any database that supports JDBC), Salesforce, and Amazon RedShift. For mobile devices, users can sync the hosted file with their FMGo or Pro clients, go off network to make changes, and sync again when a connection is available. For servers, MirrorSync allows a database to run in multiple locations, or to integrate different databases and make sure that changes to each server are reflected in the other server.
+
=== How are user accounts, value lists, and fonts handled during a deployment? ===
 +
By default accounts, value lists, and fonts will come from the Prod file during a deployment. However this is configurable. On the the layout show below, click on the settings wheel:<br>
 +
<br>
 +
[[File:Deploymentsettings.png|800px]]<br>
 +
<br>
 +
This will open the settings page where you can choose the source for accounts, value lists and fonts:<br>
 +
<br>
 +
[[File:Deploymentsettings2.png|800px]]<br>
 +
<br>
 +
Once you have set the source for these, just close the settings page using the X in the upper <b>left</b> corner.
  
MirrorSync is only installed on the server computer. It is not a plug-in, and does not require any software installation on client devices.
+
=== Deleting / adding / renaming fields ===
 +
Renaming or adding fields in the development database will work fine during the import.  If you add fields in your development copy, be aware that this field will be empty after you migrate these changes to the production server, as there will be no data on the production server to populate it.
 +
Deleting fields will work as expected, the field will be removed from the production copy.
  
MirrorSync is extremely easy to set up. The only required database changes are:
+
=== Renaming tables ===
# Enabling the FMXML and FMXDBC extended privileges in your security configuration
+
360Deploy uses the FMDataMigration tool, which will match up tables by id and name first, if no match then by name alone, then by id alone. Renaming tables will work fine as long as a new table is not created using the renamed tables previous name.
# Making sure that every table you want to sync has a primary key, modification timestamp, and creation timestamp.
 
# Creating a layout for each table.
 
# Four copy and paste operations.
 
  
<center> [[File:MirrorSyncTwitterFeed.jpg| 478 px |link=https://twitter.com/360Works |alt= @360Works on Twitter |@360Works on Twitter]] </center>
+
=== Viewing Errors ===
 +
If there is an error with a deployment, there will be a record created in the @AppLog table.  You will need to open this layout manually and scroll to the last record.  Look in the "errorDescription" field for more information about the most recent error.
  
==Why use MirrorSync?==
 
# For mobile users who access the database with FileMaker Go on their iOS device, MirrorSync allows these users to work efficiently with limited or no connectivity to the server network. Each user can take a local copy on their iPad or iPhone, input the information from the field, and then sync when they regain a network connection.  Depending on their workflow, they could even specify a one-way sync where their information is pushed to the server, but no data is pulled down, or vice versa for users who need fast read-only information.
 
# For remote users who work on laptops, MirrorSync solves the problem of how to work productively on a hosted FileMaker database, regardless of network speed. If, for example, you have multiple users connecting to FileMaker Server over a wide area network (WAN) you could notice a substantial slowdown, as every change is immediately written back to FileMaker Server. MirrorSync allows each user to take a local copy of the database which would run on the user's machine and not be affected by the WAN performance. The user could sync when they first came into the office, work on their local copy throughout the day, and sync at specified times (or on demand) to push and pull the latest data to the server.
 
# For groups of users at multiple locations, MirrorSync should be used in server-to-server mode, where an identical copy of the database is installed on FileMaker Server in each location. Each workgroup works at full LAN speed on their own FileMaker Server, and MirrorSync takes care of keeping the servers in sync with each other, so that each workgroup can see and edit changes made by the other group. In this configuration, MirrorSync should be set to sync very often, such as every 30-60 seconds.
 
#  To maximize performance and reliability, MirrorSync can be used in a clustering configuration, where two or more servers are running at the same data center and synchronized so that they all contain the same set of data. This means that each server only needs to work with 1/2 or less of the total users, and if one server goes down, the other server can take over the entire workload seamlessly. For a video demo of this, see https://www.youtube.com/watch?v=bQlGHrNtHCg
 
# For integration with non-FileMaker databases, MirrorSync is the simplest and most efficient way to accomplish this. For instance, if you have an online storefront running with MySQL or Oracle at a remote data center, but you need to have instant access to all of the orders in a database running on FileMaker Server, MirrorSync can selectively sync certain SQL tables to FileMaker, even if the table and field names do not match. In this configuration, MirrorSync should be set to sync fairly often, such as every 5-10 minutes.
 
  
== Require
+
<center>[[File:ReturntoMainPage.png| 140 px |link=http://www.360works.com/products/ |alt= 360Works Plugins for FileMaker |FileMaker Help]]</center>
<gallery>
 
[[File:CodeSnippet.png]]
 
</gallery>
 

Latest revision as of 18:51, 27 July 2023

Requirements

Overview

360Deploy is a product that simplifies the tedious process of deploying changes in a development database to an existing production database. This can be done with the click of a button while eliminating human error and reducing the downtime traditionally associated with the process. All that is required is some simple configuration, and click Deploy!

For a better understanding, consider the typical FileMaker file development scenario of a production and development server. The production server continuously hosts solutions for clients. The development/staging file is the database where alterations are being made by developers to enhance utility and better serve client's needs. Overhauling the old file to reflect the new one typically involves a process during which production is shut down and administrators sift through tables, scripts, and fields to reflect updates. We seek to bypass this arduous task by automating the entire series of events.

The entire process takes only as long as it takes to import data based on your database size, during which the databases are paused. This allows files to be updated without worrying about how often or when the changes occur. The set up consists of installing our plug-in in the FileMaker client running the deploy file, and some simple script copy/pasting.

Why use 360Deploy?

Altering live databases has traditionally been a tedious, intrusive task. It consists of server downtime and technical labor at the expense of company resources. 360Deploy is capable of doing that in minutes by seamlessly automating the process. What our software does is create a clone of the development database, transfers it to the production server, and then imports data from production into the clone to absorb the new architecture. As a safeguard, a back-up of the production file is also created on the machine before the imports occur should clients wish to revert back to the old structure.

  1. Ideal for frequent upgrades to a databases of any size
  2. Minimizes file downtime, only pausing file temporarily during import of data
  3. Drastically reduces complexity and time required to upgrade new database architecture
  4. Includes import of scripts and calculations
  5. Backs up production file in event changes want to be reverted

System requirements

  • 360Deploy requires FileMaker Pro or Advanced 16 to run the actual import process.
  • Production databases must be hosted on FileMaker Server 14 or later
  • For technical reasons, FileMaker Cloud is NOT compatible with 360Deploy

Licensing

There are four types of licenses for 360Deploy. Pricing for each license type is available at http://360works.com/360Deploy by clicking the 'pricing' button.

  • A free demo license, which can deploy to any number of servers. It has one important limitation, which is that it can only be used to deploy databases named '360Deploy Demo Solution'
  • Express Edition: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to a single deployment server
  • Enterprise License: This allows deployment of unlimited FileMaker solutions to a single deployment server. It is not legal to use this license for multiple different clients, for instance in a shared hosting server or vertical market hosted server.
  • Solution Bundle License: This allows deployment of a single FileMaker solution (this can be a multi-file solution) to unlimited deployment servers. The deployed solution can have different names for different customers.

An Enterprise License is included with the 360Works Portfolio License (http://360works.com/portfolio) at no additional charge. Any current Portfolio License holders will be able to immediately start using 360Deploy.


An important note: 360Deploys captures the list of databases when you run your first deployment for a license (and after a license reset). This is assumed to be all the files in your solution. Following that, you are allowed to deploy a subset of that list of files. If you have a large list of files in your solution, and you don't want to deploy them all at once, reach out to support@360works.com and we can hard code the list of databases against your license, so you will be able to deploy a subset of that large list of databases without issue.

Setup

View the documentation here on how to get started: 360Deploy Instructions

Linux installation instructions are included with the download but you can also view them here

Archive folder

Every time that you run a deployment, 360Deploy will put a copy of the previous production database into a timestamped directory in the FileMaker Server documents directory. This way, if anything goes wrong (due to a problem with the deployment, a programming error in the new version, or any other reason), you can simply place this timestamped archive version back into the live database directory effectively roll back to the both the structure and data that existed prior to running the deployment. You can find the archive folder at:

Mac:

/Library/360Works/Applications/MigrationBackups

Windows:

C:/Program Files/360Works/Applications/MigrationBackups

Keep in mind that this archive directory is never automatically deleted, and can potentially consume a lot of disk space if your solution is large. It's a good idea to manually clean up this directory periodically.

Also keep in mind that if you are using containers with external storage, that container data is shared between the old and new database versions - there is no separate copy placed into the archive folder. This speeds up the import and conserves disk space, but it also means that if you do need to roll back the database version, changes that were made to the external container data are not rolled back. Use FileMaker Server's built-in backup feature to guarantee a complete backup including a separate copy of all container data.

Scheduling after-hour imports

The script "Upload Files and Start Migration ( $~configuration_id )" is Server Side compatible, and can be used in a Server Side Schedule, which is our recommended way of doing a scheduled deployment. To use this script, we recommend creating a wrapper script, where you can pass in the id of the configuration to the "Upload Files and Start Migration" script.

This script does make use of the 360Deploy plugin, so you will need to install that server side if you intend to schedule deployments.

See the example script in the 360Deploy.fmp12 file called "PSOS : 360Store Deployment"

To get the id of your configuration, open the data viewer and evaluate the "Configuration::__id" field while you are on the main layout. This will be the value you pass in.

External Data Sources Are Different Between Dev and Prod

In a multi file solution there will typically be External Data Sources. Now if you have a Dev environment with Dev External Data Sources named one way, and a Prod environment with Prod External Data Sources named a different way, this situation requires a little attention before it is 360Deploy ready. If you did nothing before running a deployment, your Dev file would be deployed into the Prod environment, but would still be looking for the Dev External Data Sources (which will either not be found, or perhaps worse, actually resolve correctly but now you are using the wrong database).

The solution to this problem is to use Dynamic Data Sources. Dynamic Data Sources allow us to set an External Data Source to a Global Variable, then set that Global Variable during the Startup script. Whatever we set that variable to is what FileMaker will use to resolve the data source. View more documentation about Dynamic Data Sources here: Dynamic Data Sources

Dynamic Data Sources allow us to solve this problem of different data sources between Dev and Prod. First, change your data sources to Dynamic Data Sources. Use a Global Variable for each of your data sources. Then in your startup script, create an if/else branch that will determine whether we are running in the Prod or Dev environment, then set the Global Variable to the appropriate path for your data source. There are two common approaches here:

You can use a naming convention that allows you to programmatically determine the right external data source name.

So for example, let's say I name my main file "MySolution", I would name the external data sources after it: "MySolution_Invoices" and "MySolution_Inventory". In my startup script, I can determine the proper external data source by doing: Set Variable: $$invoicesExternalDataSource: Get(FileName) & "_Invoices"

Alternatively, and a little bit less clear, is you can take advantage of the fact that 360Deploy preserves the data from the prod file. So you could store the name of the external data source inside a field in the prod file, and have your startup script reference that field when setting up the dynamic data source. This approach is more flexible, because you could have different files follow different naming conventions for their external data sources, but it's a little harder to follow.


Miscellaneous tips

How are user accounts, value lists, and fonts handled during a deployment?

By default accounts, value lists, and fonts will come from the Prod file during a deployment. However this is configurable. On the the layout show below, click on the settings wheel:

Deploymentsettings.png

This will open the settings page where you can choose the source for accounts, value lists and fonts:

Deploymentsettings2.png

Once you have set the source for these, just close the settings page using the X in the upper left corner.

Deleting / adding / renaming fields

Renaming or adding fields in the development database will work fine during the import. If you add fields in your development copy, be aware that this field will be empty after you migrate these changes to the production server, as there will be no data on the production server to populate it. Deleting fields will work as expected, the field will be removed from the production copy.

Renaming tables

360Deploy uses the FMDataMigration tool, which will match up tables by id and name first, if no match then by name alone, then by id alone. Renaming tables will work fine as long as a new table is not created using the renamed tables previous name.

Viewing Errors

If there is an error with a deployment, there will be a record created in the @AppLog table. You will need to open this layout manually and scroll to the last record. Look in the "errorDescription" field for more information about the most recent error.


360Works Plugins for FileMaker