Generate top 50 popular resources of AWS using 100 lines of code
KubeVela currently supports AWS, Azure, GCP, AliCloud, Tencent Cloud, Baidu Cloud, UCloud and other cloud vendors, and also provides a quick and easy command line tool to introduce cloud resources from cloud providers. But supporting cloud resources from cloud providers one by one in KubeVela is not conducive to quickly satisfying users' needs for cloud resources. This doc provides a solution to quickly introduce the top 50 most popular cloud resources from AWS in less than 100 lines of code.
We also expect users to be inspired by this article to contribute cloud resources for other cloud providers.
#
Where are the most popular cloud resources on AWS?The official Terraform website provides Terraform modules for each cloud provider, for example, AWS cloud resource Terraform modules. And the cloud resources are sorted by popularity of usage (downloads), for example, AWS VPC has 18.7 million downloads.
Through a simple analysis, we found that the data for the top 50 popular Terraform modules for AWS can be obtained by requesting https://registry.terraform.io/v2/modules?filter%5Bprovider%5D=aws&include=latest-version&page%5Bsize%5D=50&page%5Bnumber%5D=1.
#
PrerequisitesThe code accepts two parameters.
- provider Name
- The URL of the Terraform Modules corresponding to the provider
For AWS, Provider Name should be “aws”,corresponding Terraform modules URL isTerraform Modules json API(Searching top 50 popular resources for provider aws in Terraform Registry).
You need to make sure the providerName(aws) and Modules links are correct before executing the code.
#
Executing the codeThen you can quickly bring in the top 50 most popular AWS cloud resources in bulk with the following 100 lines of code (filename gen.go).
Executing the following command:
#
Explanation for the code#
Unmarshal the json data for the resourcesAccess the URL passed in by the user and parse the returned json data into the Go structure.
The json format corresponding to the resource is as follows.
In the json data corresponding to Modules, we only care about two key-value pairs, viz.
- data: A list containing the names and properties of Modules
- Included: Information about the specific version of Modules filtered out
In this case, for each Module element in data, resolve its attributes, Id and the id corresponding to the latest-version in relationship; for each Module version element in Included, resolve its attributes and Id.
The attributes are further resolved as follows five items:
- Name
- Downloads
- Source
- Description
- Verified
The Go structure is named as TFDownload
, The http library gets the json data and then parses the structure of the Terraform modules through the json.Unmarshal
.
#
generating ComponentDefinitions in batch- creating directory and component definitions
After parsing, create a new folder in the current directory and name the folder as \<provider name>.
Iterate through the parsed data, and for each Module element, perform the following operations to generate the corresponding definition and documentation for it.
- Generate definition files
Generate the definition file by reading the corresponding information from the module's github repository using the following vela command.
Several items to be filled in the instruction are passed in from the parsed Module structure.
- gitURL: {Module.Attributes.Source}.git
- description: If there are elements in
Included
which have the same ID with relationship.latest-version.ID, set the description as the corresponding description inIncluded
elements, otherwise set the description as providerName+ModuleName. - yamlFileName:terraform-{providerName}-{Module.Attributes.Name}.yaml
#
Have a try?There are also a number of cloud providers that offer a wealth of Terraform modules, such as
GCP: https://registry.terraform.io/namespaces/terraform-google-modules
Alibaba Cloud: https://registry.terraform.io/namespaces/terraform-alicloud-modules
Do you want to extend cloud resources for your current or favorite cloud provider for KubeVela as well?