Syndicate deployment framework
Syndicate
aws-syndicate is an Amazon Web Services deployment framework written in Python,
which allows to easily deploy serverless applications using resource
descriptions.
Changelog
1. Installation
1.1 Installation from PyPI
Install aws-syndicate from PyPI
1.2 Installation from source code
1.2.1 Prerequisites
- Python 3.10 version;
- Package manager PIP 22.0
or higher version; - Apache Maven 3.3.9
or higher version (for Java projects).
1.2.2 Installation on macOS
Detailed guide how to install Python you can find
here.
Also here
you can find detailed guild how to install
the latest Apache Maven.
- Pull the project
git clone https://github.com/epam/aws-syndicate.git
- Create virtual environment:
python3 -m venv syndicate_venv
- Activate your virtual environment:
source syndicate_venv/bin/activate
- Install Syndicate framework with pip from GitHub:
pip3 install aws-syndicate/.
- Set up a Syndicate Java plugin:
mvn install -f aws-syndicate/plugin/
- Go to the
2. Usage guide
1.2.2 Installation on Linux
Detailed guide how to install Python you can
find here.
Also here
you can find detailed guild how to install
the latest Apache Maven.
- Pull the project
git clone https://github.com/epam/aws-syndicate.git
- Create virtual environment:
python3 -m venv syndicate_venv
- Activate your virtual environment:
source syndicate_venv/bin/activate
- Install Syndicate framework with pip from GitHub:
pip3 install aws-syndicate/.
- Set up a Syndicate Java plugin:
mvn install -f aws-syndicate/plugin/
- Go to the
2. Usage guide
1.2.3 Installation on Windows
Detailed guide how to install Python you can
find here.
Also here
you can find detailed guild how to install
the latest Apache Maven.
- Pull the project
git clone https://github.com/epam/aws-syndicate.git
- Create virtual environment:
python3 -m venv syndicate_venv
- Activate your virtual environment:
syndicate_venv\Scripts\activate.bat
- Install Syndicate framework with pip from GitHub:
pip3 install aws-syndicate/.
- Set up a Syndicate Java plugin:
mvn install -f aws-syndicate/plugin/
- Go to the
2. Usage guide
2. Usage guide
2.1 Creating Project files
Execute syndicate generate project command to generate the project with all the
necessary components and in a right folders/files hierarchy to start developing
in a min. Command example:
syndicate generate project
--name $project_name
--config_path $path_to_project
All the provided information is validated. After the project folder will be
generated the command will return the following message:
Project name: $project_name
Project path: $path_to_project
The following files will be created in this folder: .gitignore, .syndicate,
CHANGELOG.md, deployment_resources.json, README.md.
Command sample:
syndicate generate project --name DemoSyndicateJava && cd DemoSyndicateJava
For more details please execute syndicate generate project --help
2.2 Creating configuration files for environment
Execute the syndicate generate config command to create Syndicate configuration
files. Command example:
syndicate generate config
--name $configuration_name [required]
--region $region_name [required]
--bundle_bucket_name $s3_bucket_name [required]
--access_key $access_key
--secret_key $secret_key
--config_path $path_to_store_config
--project_path $relative_path_to_project
--prefix $prefix
--suffix $suffix
--extended_prefix $extended_prefix_mode
--use_temp_creds $use_temp_creds #Specify,if use mfa or access_role
--access_role $role_name
--serial_number $serial_number
--tags $KEY:VALUE
--iam_permissions_boundary $ARN
All the provided information is validated.
Note: you may not specify --access_key and --secret_key params. In this
case Syndicate will try to find your credentials by the path ~/.aws.
Note: You can force Syndicate to generate temporary credentials and use them
for deployment. For such cases, set use_temp_creds parameter to True and
specify serial number if IAM user which will be used for deployment has a policy
that requires MFA authentication. Syndicate will prompt you to enter MFA code to
generate new credentials, save and use them until expiration.
After the configuration files will be generated the command will return the
following message:
Syndicate initialization has been completed.
Set SDCT_CONF:
Unix: export SDCT_CONF=$path_to_store_config
Windows: setx SDCT_CONF $path_to_store_config
Just copy one of the last two lines, depending on your OS, and execute the
command. The commands set the environment variable SDCT_CONF required by
aws-syndicate to operate.
Pay attention that the default syndicate_aliases.yaml file has been generated.
Your application may require additional aliases to be deployed - please add them to the file.
Command sample:
SYNDICATE_AWS_ACCESS_KEY=# enter your aws_access_key_id here
SYNDICATE_AWS_SECRET_KEY=# enter your aws_secret_access_key here
syndicate generate config --name dev --region eu-central-1 --bundle_bucket_name syndicate-artifacts-eu-central-1 --access_key $SYNDICATE_AWS_ACCESS_KEY --secret_key $SYNDICATE_AWS_SECRET_KEY --config_path $(pwd) --prefix syn- --suffix -dev --tags ENV:DEV
For more details please execute syndicate generate config --help
2.3 Creating lambda files
Execute syndicate generate lambda command to generate required environment for
lambda function except business logic. Command example:
syndicate generate lambda
--name $lambda_name_1
--runtime python|java|nodejs
--project_path $project_path
All the provided information is validated. Different environments will be
created for different runtimes:
- for Python
.
├── $project_path
│ └── src
│ ├── commons
│ │ ├── __init__.py
│ │ ├── abstract_lambda.py
│ │ ├── exception.py
│ │ └── log_helper.py
│ └── lambdas
│ ├── $lambda_name_1
│ │ ├── __init__.py
│ │ ├── deployment_resources.json
│ │ ├── handler.py
│ │ ├── lambda_config.json
│ │ ├── local_requirements.txt
│ │ └── requirements.txt
│ ├── $lambda_name_2
│ │ ├── __init__.py
│ │ ├── deployment_resources.json
│ │ ├── handler.py
│ │ ├── lambda_config.json
│ │ ├── local_requirements.txt
│ │ └── requirements.txt
│ ├── __init__.py
│ └── ...
└── ...
- for Java
.
├── $project_path
│ └── jsrc
│ └── main
│ └── java
│ └── com
│ └── $projectpath
│ ├── $lambda_name_1.java
│ └── $lambda_name_2.java
└── ...
- for NodeJS
.
├── $project_path
│ └── app
│ └── lambdas
│ ├── $lambda_name_1
│ │ ├── deployment_resources.json
│ │ ├── lambda_config.json
│ │ ├── index.js
│ │ ├── package.json
│ │ └── package-lock.json
│ └── $lambda_name_2
│ ├── deployment_resources.json
│ ├── lambda_config.json
│ ├── index.js
│ ├── package.json
│ └── package-lock.json
└── ...
Command sample:
syndicate generate lambda --name DemoLambda --runtime java --project_path $(pwd)
For more details please execute syndicate generate lambda --help
2.4 Add other infrastructure components
This step is optional and could be skipped while getting familiar with syndicate.
All the resources syndicate works with could be generated in the same way as lambda.
Invoke the syndicate generate meta --help command to find out which resources are available.
2.5 Open project in your IDE
Now the project is ready to be adjusted. Consider opening your favourite IDE and observe the files created by syndicate.
3. Deployment
3.1 Create an S3 bucket for aws-syndicate artifacts:
syndicate create_deploy_target_bucket
3.2 Build project artifacts
syndicate build
3.3 Deploy project resources to AWS account
syndicate deploy
Now the DemoLambda is created and available to be tested.
3.4 Update resources
In order to be sure your latest changes works well on the AWS account the application should be deployed to the AWS account.
To do this use the following commands set:
syndicate build
syndicate update
3.5 Lambdas invocations metrics
syndicate profiler
3.6 Clean up project resources from AWS Account
syndicate clean
3.7 Observing the environment manipulation history
syndicate status # this shows the general CLI dashboard where latest modification, locks state, latest event, project resources are shown
syndicate status --events # this returns all the history of what happened to the environment
4. Examples
If you are just getting familiar with the functionality, you can use one of the
pre-prepared examples that contain a minimum set of AWS resources and lambdas.
The aws-syndicate/examples folder contains structure examples for different
runtimes. Go to any example you like best and set the environment
variable SDCT_CONF=$path_to_the_selected_example.
Add your account details to syndicate.yml file - account id,
secret access key, access key and bucket name for deployment.
To syndicate_aliases.yml add your account id, region
name (eu-central-1, us-west-1, etc.) and other values in the file that start
with a $ sign.
Documentation
Documentation you can find here
Getting Help
We use GitHub issues for tracking bugs and feature requests. You can find our
public backlog here. If it
turns out that you may have found a bug,
please open an issue
with some of existing templates.
Default label for bugs, improvements and feature requests is To-Think-About,
it defines that ticket requires additional information about what should be done
in scope of this issue.
To-Do label should be added only for tickets with clear and reviewed issue
scope.
But before creating new issues - check existing, they may cover your problem or
question. For increasing issue priority - just add “+1” comment.
Would like to contribute?
Please,
check contributor guide
before starting.
