AutoSys jobs deployment and versioning

Problem description


 You are using AutoSys job scheduler to manage workflow of your application. To define them, you are using jobs defined in jil language. During development, you are using different boxes than in UAT stage or production. That means, you somehow have to manage lifecycle of your Jils to make sure you could move smoothly between different stages of development lifecycle.

Deployment

Deployment is build around scripts and mapping files, which results in refreshed AutoSys autos box and log information about scripts processing state.

1. Init script.


 

AutoSys_Deployment_init
Init script is responsible for taking environment name. Then it invokes in order action scripts passing to them given environment name.
  • Inputs – environment name (DEV1, DEV2, DEV3, UAT1, UAT2, PROD ….)
  • Outputs – log file

2. Cleanup script.


 

 AutoSys_Deployment_cleanup
Cleanup script based on environment name, identifies mapping file with patterns for finding jobs to cleanup. Using found pattern, it deletes all jils on given AutoSys box.
 
  • Prerequisites –  mapping file with patterns per environment to find jobs for cleanup.
  • Inputs – environment name (DEV1, DEV2, DEV3, UAT1, UAT2, PROD ….)
  • Outputs – log file

3. Compile script.


 

 AutoSys_Deployment_compile
Compile script takes the jils template file, then using given environment key it locates environment context file which could be injected into template. Finally, it produces output jil file in path related to environment.
 
  • Prerequisites
    • jil template file (all environment related things are descoped)
    • set of context files which are used for filling template per environment
  • Inputs – environment name (DEV1, DEV2, DEV3, UAT1, UAT2, PROD ….)
  • Outputs
    • log file
    • jils script dedicated for given environment

4. Run script.


 

 AutoSys_Deployment_run
Based on env key it locates compiled script  and run all jils on specified AutoSys box.
 
Destination AutoSys box could be identified by convention using the same environment name. That could be coded as convention (any DEV* -> DEV11, any UAT* -> UAT11, PROD -> PROD11).
 
  • Prerequisites – jils script dedicated for given environment
  • Inputs – environment name (DEV1, DEV2, DEV3, UAT1, UAT2, PROD ….)
  • Outputs – log file

5. Diff script.


 

 AutoSys_Deployment_diff
Diff script is kind of reconciliation process, which if previous step was success generates difference files for comparison with PROD (or any other specified) environment. That should produce set of three kind of differences:
  1. exists on ENV, not on PROD;
  2. exists on PROD, not on ENV;
  3. exists on BOTH, but with differences.
 
  • Prerequisites – jils successfully applied to AutoSys Box
  • Inputs – environment name (DEV1, DEV2, DEV3, UAT1, UAT2, PROD ….)
  • Outputs
    • log file
    • diff reports

General Overview


 

AutoSys_Deployment
 

Versioning


 

Considering above, it is good practice to keep below artifacts under favourite Version Control system (git of course):

  • All Input files – cleanup patterns, jils template, environment context files
  • All Scripts – init, cleanup, compile, run and diff
  • Jils snapshots, at least production one (Jils Output)
  • (Optional) – diff scripts used to apply patches, updates or any changes on production

Summary


 

That’s only proof of concept, which could be implemented in any language. Some changes could be applied to overall design based on specific needs, but generally speaking approach should work for most of the cases.

There is still couple of open topics:

  • monitoring of log files, automatic vs manual, how to approach auto monitoring in continuous integration
  • running of above process from CI server, triggered by changes into sources (input files, scripts)
  • proper decomposition of template file from any environment dependent settings
  • better specified diff script, what compare to what, which parts of jils should be compared, etc
  • running scripts on AutoSys boxes, security considerations
  • how to apply deployment on production, having in mind possible restricted policies

[Sources]

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s