Policy Authoring Walkthrough
The following tutorial walks through how to develop a custom policy using the fpt command-line tool to demonstrate the general approach and utilities that can be used when developing custom policies.
It is recommended you store policy template code in a source control system, such as github, with the extension .pt. Policies can be developed in any text editor and then uploaded and applied to an account using the policy manager user interface or the fpt tool.
For this example, we're going to create a policy that identifies and deletes unattached IP addresses. On AWS, these correspond to elastic IPs and cost money even when not in use. These type of resources can often accumulate if left unchecked and run up cost unnecessarily.
Creating this sample policy involves the following steps:
| • | Determining Inputs |
| • | Gathering Data |
| • | Processing Data |
| • | Adding the Validations |
| • | Defining the Escalation Actions |
As a policy designer, you determine which user inputs you need to effectively deploy a policy.
This sample policy accepts two parameters:
| • | param_allow_list: IP addresses to not consider for deletion |
| • | param_email: who to email reports to |
The initial policy code we develop includes the needed inputs. Then we call any needed APIs to gather the data for this policy. Starting with this step ensures that data is being retrieved as expected and saves the validation and actions until we ensure our data is correct.
name "Unattached IP Addresses"
rs_pt_ver 20180301
type "policy"
short_description "Cleanup Unattached IP addresses"
long_description "Checks for Unattached IP Addresses and automatically deletes them."
severity "low"
category "Cost"
permission "perm_read__ip_addresses" do
actions "rs_cm.show","rs_cm.index"
resources "rs_cm.ip_addresses","rs_cm.clouds"
end
parameter "param_allow_list" do
type "list"
label "White list of IP Addresses to keep"
end
resources "clouds", type: "rs_cm.clouds"
resources "addresses", type: "rs_cm.ip_addresses" do
iterate @clouds
cloud_href href(iter_item)
end
policy "unattached_ip_addresses" do
validate_each @addresses do
summary_template "Unattached IP addresses to delete"
detail_template <<-EOF
# Unattached IP Addresses
{{ range data -}}
{{ end -}}
EOF
# check ... TBD -- set to false for now
check false
end
end
To pass syntax checks and ensure that an incident is created with our data, we used a check false in the validate_each statement, which we'll come back to later to provide a real check.
After saving the file as tutorial.pt you can run it with fpt run tutorial.pt param_allow_list=. Running the command outputs the debug log, which shows a list of API calls as the policy executes. If you have any IP addresses in this account, you should see them here. If not, you can create them in the Flexera One Cloud Dashboard.
You can run the policy again to see how results have updated after adding more IP addresses. Or you can do a fpt retrieve_data tutorial.pt param_allow_list= to download the data to disk for use in later commands. The retrieve_data command runs the policy and extracts the datasource data. For this example it will output resources_clouds.json and resources_addresses.json and save them.
Here's a sample API response with one sample unattached IP and one sample attached IP:
[
{
"address": "10.20.30.50",
"created_at": "2018/05/07 22:00:12 +0000",
"domain": "ec2_classic",
"links": [
{
"href": "/api/clouds/1/ip_addresses/B41PVJI70FMLO",
"rel": "self"
},
{
"href": "/api/networks/98IF6IJR1ERIB",
"rel": "network"
}
],
"name": "unattached_ip_address",
"updated_at": "2018/05/07 22:00:12 +0000"
},
{
"address": "10.20.30.40",
"created_at": "2014/10/03 17:17:34 +0000",
"domain": "ec2_classic",
"links": [
{
"href": "/api/clouds/1/ip_addresses/4C25RVT8G7U3B",
"rel": "self"
},
{
"href": "/api/networks/C72HP153S3UOF",
"rel": "network"
},
{
"href": "/api/deployments/454876003",
"rel": "deployment"
},
{
"href": "/api/clouds/1/ip_addresses/4C25RVT8G7U3B/ip_address_bindings",
"rel": "ip_address_bindings"
}
],
"name": "attached_ip_address",
"updated_at": "2016/04/05 05:23:04 +0000"
}
]
Save these two sample API addresses to a file named mock_addresses.json.
Note how an IP address attached to an instance has an entry in the links array of type "rel": "ip_address_bindings". An IP address which is not attached has no entry. Therefore, this policy wants to then check that all entries in the datasource have "rel": "ip_address_binding" entry present, indicating they are all attached and in-use.
While we could have a complicated set of extraction logic in the validate_each statement, it would be easier to do the logic in JavaScript and make a new variable is_attached that can be checked with simple equals statement. Add and save the existing script to your tutorial.pt:
datasource "ds_munged_addresses" do
run_script $js_munge_addresses, @addresses, $param_allow_list
end
script "js_munge_addresses", type: "javascript" do
parameters "addresses", "ip_allow_list"
result "unattached_ips"
code <<-EOS
var unattached_ips = []
// rs_href extracts a hrefs for a RightScale API 1.5 resource
// hrefs are stored in an array of link objects with values "rel" and "href"
function rs_href(res, rel) {
for (var j = 0; j < res["links"].length; j++) {
if (res["links"][j]['rel'] == rel) {
return res["links"][j]['href']
}
}
return ""
}
console.log("iterating over addresses, count:", addresses.length)
for (var i = 0; i < addresses.length; i++) {
var address = addresses[i]
is_attached = false
self_href = rs_href(address, "self")
binding_href = rs_href(address, "ip_address_bindings")
var is_excluded = ip_allow_list.indexOf(address['address']) >= 0
var is_attached = binding_href != ""
unattached_ips.push({
address: address['address'],
name: address['name'],
href: self_href,
attached: is_attached,
excluded: is_excluded,
})
}
EOS
end
Note:Note the following about the script:
| • | It has two inputs for addresses obtained earlier and an optional safelist of IPs. When running the script from the command line all options must be supplied. Since safelist is an array supply you just supply a JSON array when supplying this parameter. |
| • | It has one output called unattached_ips. This is the name of the variable that will turn into the script. |
| • | We can also define functions internally and use them later. Functions can also easily be pasted into your browsers JavaScript console to develop a bit more interactively. |
| • | It has a console.log statement for printing out interim results. |
Running fpt script tutorial.pt addresses=@mock_addresses.json ip_allow_list=[]
Gets output:
Running script "js_munge_addresses" from tutorial.pt and writing unattached_ips to out.json
console.log: "iterating over addresses, count:" 2
JavaScript finished, duration=457.244µs
Which outputs:
[
{
"address": "10.20.30.50",
"attached": false,
"excluded": false,
"href": "/api/clouds/1/ip_addresses/B41PVJI70FMLO",
"name": "unattached_ip_address"
},
{
"address": "10.20.30.40",
"attached": true,
"excluded": false,
"href": "/api/clouds/1/ip_addresses/4C25RVT8G7U3B",
"name": "attached_ip_address"
}
]
Excellent! We have simplified the output to show only what we need and added an explicit field to indicate whether the IP is attached or not.
With our processed data, the policy logic validation logic is fairly easy to write. The record should have attached==true. If it isn't attached, then it's in violation unless excluded==true. Let's add the validation logic to check for those conditions.
policy "unattached_ip_addresses" do
validate_each $ds_munged_addresses do
summary_template "{{ rs_project_name }} (Account ID: {{ rs_project_id }}): {{ len data }} Unattached IP Addresses"
detail_template <<-EOS
# Unattached IP Addresses
| Name | Address | Href |
| ---- | ------- | ---- |
{{ range data -}}
| {{.name}} | {{.address}} | {{.href}} |
{{ end -}}
EOS
check logic_or(
val(item, 'attached'),
val(item, 'excluded')
)
end
end
Try out passing in different parameters such as fpt script tutorial.pt addresses=@mock_addresses.json ip_allow_list=["10.20.30.50"] and seeing how the output changes. Run the script to see the final table of entries printed out by the detail_template block above.
Defining the Escalation Actions
Finally add the escalation and save the file. Data is equal to all the IP addresses violating the policy. This means that you don't have to check for the excluded or attached fields and can just destroy everything passed in.
escalation "delete_addresses" do
request_approval do
label "Delete IP"
description "If approved, automatically deleted unattached IP address"
parameter "approval_reason" do
type "string"
label "Reason for approval"
description "Explain why you are approving the action"
end
end
run "delete_unattached_addresses", data
end
define delete_unattached_addresses($data) do
foreach $item in $data do
@ip_address = rs_cm.get(href: $item['href'])
@ip_address.destroy()
end
end
The policy block should get a reference to the escalation:
policy "unattached_ip_addresses" do
....
escalate $delete_addresses
end
By default actions are not run. You can pass -r to the fpt run command to run things. Any code in a define block will get run as a Cloud Workflow, a custom orchestration language. The Cloud Workflow Language is powerful and outside the scope of this walk-through.