Signed URLs #AWS #ansible filters

Based on “Sharing #AWS S3 files for ansible thru Signed URLs” and URL split filter for ansible

I was editing my config file to download images thru ansible once or twice a week. Time to automate that.

the filter

Please check my previous post to see what do you need to prepare this. Actually, I have everything in a container. In summary, just need “aws” and “boto” installed thru python-pip

# {% set credentials = [ aws_key, aws_pass, aws_bucket] %}
# {% set aws_vars = [ release_base_dir , '.*Nuage-elastic-.*\.qcow2$'] | join() | aws_s3_list(credentials) | signed_url_7d(credentials) %}
#
import sys
import boto
from boto.s3.key import Key
import requests
import re

class FilterModule (object):
    def filters(self):
        return {
            "signed_url_7d"   :   self.signed_url_7d,
            "aws_s3_list"   :   self.aws_s3_list,
        }

    def signed_url_7d(self, path , credentials ):
        s3_session   =   boto.connect_s3(credentials[0], credentials[1])
        s3_bucket    =   s3_session.get_bucket(credentials[2], validate=False)
        s3_key       =   Key(s3_bucket)
        s3_key.key   =   path
        result = {
            'url'    : s3_key.generate_url( expires_in=604800 )
        }
        return result

    def aws_s3_list(self, reg_value, credentials ):
        s3_session   =   boto.connect_s3(credentials[0], credentials[1])
        s3_bucket    =   s3_session.get_bucket(credentials[2], validate=False)
        regexp       =   re.compile(reg_value)
#        return reg_value
        for key in s3_bucket.list():
             if regexp.search(key.name.encode('utf-8')):
                   return key.name.encode('utf-8')
        return 'ERROR: regex not found'

What do we want?

This is the file we will try to get to download my tar and qcow2 images.

url_base_location: "https://nuage-secure-files.s3.amazonaws.com/5.2.2-files/"
#files release dependant
url_elasticsearch_qcow2: "Nuage-elastic-5.2.2-22.qcow2?Signature=Z7g3C%2BOcAM2zPEjZ17Smyqbtt4E%3D&Expires=1521557331&AWSAccessKeyId={{ aws_access_key }}"
url_openvswitch_rpm: "nuage-openvswitch-5.2.2-25.el7.x86_64.rpm?Signature=ZdLX2sCZHKUcLDbx177V0Ghiing%3D&Expires=1521557332&AWSAccessKeyId={{ aws_access_key }}"
url_vsc_qcow2: "vsc_singledisk.qcow2?Signature=2C3oIIz0Kx%2BV0RO32I8SB7bg5JQ%3D&Expires=1521557333&AWSAccessKeyId={{ aws_access_key }}"
url_vsd_qcow2: "VSD-5.2.2_24.qcow2?Signature=U171si2fTXBqeXe9C%2B26H7dN4gw%3D&Expires=1521557333&AWSAccessKeyId={{ aws_access_key }}"

What template do we’ll use

Here you have the template in the folder templates in the playbook dir:

{% set credentials = [ aws_key, aws_pass, aws_bucket] %}
{% set aws_vars = [ release_base_dir , '.*Nuage-elastic-.*\.qcow2$'] | join() | aws_s3_list(credentials) | signed_url_7d(credentials) %}

url_base_location: "{{ 'https://' + aws_vars.url | urlsplit('hostname') + '/' + release_base_dir + '/' }}"

{% set aws_vars = [ release_base_dir , '.*Nuage-elastic-.*\.qcow2$'] | join() | aws_s3_list(credentials) | signed_url_7d(credentials) %}
url_elasticsearch_qcow2: "{{ aws_vars.url | replace( 'https://' + aws_vars.url | urlsplit('hostname') + '/' + release_base_dir + '/' , '') | replace( aws_key , '{'+'{'+' aws_access_key '+'}'+'}')  }}"
{% set aws_vars = [ release_base_dir , '.*nuage-openvswitch-.*\.rpm$'] | join() | aws_s3_list(credentials) | signed_url_7d(credentials) %}
url_openvswitch_rpm: "{{ aws_vars.url | replace( 'https://' + aws_vars.url | urlsplit('hostname') + '/' + release_base_dir + '/' , '') | replace( aws_key , '{'+'{'+' aws_access_key '+'}'+'}')  }}"
{% set aws_vars = [ release_base_dir , '.*vsc_singledisk.*\.qcow2$'] | join() | aws_s3_list(credentials) | signed_url_7d(credentials) %}
url_vsc_qcow2: "{{ aws_vars.url | replace( 'https://' + aws_vars.url | urlsplit('hostname') + '/' + release_base_dir + '/' , '') | replace( aws_key , '{'+'{'+' aws_access_key '+'}'+'}')  }}"
{% set aws_vars = [ release_base_dir , '.*VSD-.*\.qcow2$'] | join() | aws_s3_list(credentials) | signed_url_7d(credentials) %}
url_vsd_qcow2: "{{ aws_vars.url | replace( 'https://' + aws_vars.url | urlsplit('hostname') + '/' + release_base_dir + '/' , '') | replace( aws_key , '{'+'{'+' aws_access_key '+'}'+'}')  }}"

the playbook and how it works

This is the playbook

- hosts: localhost
  remote_user: root
  tasks:
    - name: test jinja2
      template: src=files_location.j2 dest="{{ playbook_dir }}/cfg/{{ cfg_dir }}/files_location.yml" backup=no

Let’s run this way (replace aws_key and aws_pass key with your own):

ansible-playbook -e aws_key="ABCDILUSFSBXLXMIQRSA" -e aws_pass="ABCD*T2oRghq47TCIKxoH2lhk4XXMibLkfilih/" -e aws_bucket="nuage-secure-files" -e release_base_dir="5.2.2-files" -e cfg_dir=5.2.2-docker-sdwan-elastic-ip  files-location.yml

See ya!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s