阿里云实践

# 一、实践概述

# Terraform阿里云实践

# 云产品资源

网络
- DNS
- VPC
- EIP
负载均衡 SLB
弹性计算 ECS
对象存储 OSS
表格存储 Tablestore

# 二、Terraform初始化配置

# 配置创建backend资源

## 进入到backend目录
cd terraform-alicloud-operator/global/backend

## vim main.tf
provider "alicloud" {
  access_key = var.alicloud_access_key
  secret_key = var.alicloud_secret_key
  region     = var.region
}

## OSS
resource "alicloud_oss_bucket" "terra-backend" {
  bucket = "terraform-backend-data-202209"
  acl    = "private"
}

## tablestore
resource "alicloud_ots_instance" "terra-table" {
  name        = "terra-table"
  description = "terraform tablestore"
  accessed_by = "Any"
  tags = {
    Created = "TF"
    For     = "Building table"
  }
}

resource "alicloud_ots_table" "basic" {
  instance_name = alicloud_ots_instance.terra-table.name
  table_name    = "terra_table"
  primary_key {
    name = "LockID"
    type = "String"
  }
  time_to_live                  = -1
  max_version                   = 1
  deviation_cell_version_in_sec = 1
}

## vim variables.tf
# variable "alicloud_access_key" {
#   type = string
# }

# variable "alicloud_secret_key" {
#   type = string
# }

variable "region" {
  type        = string
  description = "region name"
  default     = "cn-shanghai"
  sensitive   = true
}

# 格式terraform文件
terraform fmt


terraform init -plugin-dir /root/.terraform.d/terraform-plugin-cache

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

# Terrraform本地缓存配置

这里利用项目先进行初始化下载alicloud的插件缓存到指定目录中

这里是在CentOS系统进行配置的

避免插件因为网络问题下载失败,提前配置好插件缓存. 这里通过TF_CLI_CONFIG 变量指定Terraform CLI的配置文件

# 项目目录中创建`.terraformrc`文件
$ cd youdianzhishi-terraform/terraform-alicloud-operator
$ touch ./.terraformrc

## 创建编辑terraform本地缓存的配置文件 
$ vim .terraformrc
plugin_cache_dir  = "$HOME/.terraform.d/terraform-plugin-cache"
disable_checkpoint = true


## 需要手动创建plugin的缓存路径
$ mkdir -pv /root/.terraform.d/terraform-plugin-cache

## 声明 TF_CLI_CONFIG路径
$ export TF_CLI_CONFIG_FILE=/home/terraform/youdianzhishi-terraform/terraform-alicloud-operator/.terraformrc

## 声明认证配置的环境变量(这里要结合上面的`TF_CLI_CONFIG_FILE`一起使用,好让terraform的plugin缓存到指定路径中)
export ALICLOUD_ACCESS_KEY="xxxxxx"
export ALICLOUD_SECRET_KEY="xxxxxx"
export ALICLOUD_REGION="cn-shanghai"

## 进入到`global/backend`项目路径下,初始化并下载terraform plugin到指定缓存目录
$ cd terraform-alicloud-operator/global/backend
$ terraform  init 

Initializing the backend...

Initializing provider plugins...
- Finding aliyun/alicloud versions matching "1.183.0"...
- Installing aliyun/alicloud v1.183.0...  # 这里此时是联网下载`alicloud`的plugin组件的
- Installed aliyun/alicloud v1.183.0 (signed by a HashiCorp partner, key ID 47422B4AA9FA381B)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.


## 给`TF_CLI_CONFIG` 文件添加,走本地缓存目录寻找plugin组件配置
$ vim /home/terraform/youdianzhishi-terraform/terraform-alicloud-operator/.terraformrc
provider_installation {
  filesystem_mirror {
    path    = "/root/.terraform.d/terraform-plugin-cache"
    include = ["registry.terraform.io/*/*"]
  }
}


## 再次terraform init初始化,terraform没到远程下载plugin
$ terraform  init 

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of aliyun/alicloud from the dependency lock file
- Using previously-installed aliyun/alicloud v1.183.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

plugin_cache_dir是插件的缓存目录(此目录需提前创建不然init会报错)
disable_checkpoint禁用需要连接HashiCorp体统的网络服务的升级和安全公告检查

# 备注:

初始化插件下载有两种方式:

1.通过terraform init自动下载provider插件
但是在.terraformrc文件中不能添加如下配置内容,否则就不是初始化下载插件;而是等于在指定缓存路径寻找并使用provider插件
```
provider_installation {
  filesystem_mirror {
    path    = "/root/.terraform.d/terraform-plugin-cache"
    include = ["registry.terraform.io/*/*"]
  }
}
```
1
2
3
4
5
6
2.登入register.terraform.io手动到GitHub下载,并按照目录结构放到plugin_cache_dir;

# 操作创建阿里云backend的资源

# 进入到backend项目路径
$ cd youdianzhishi-terraform/terraform-alicloud-operator/global/backend

# 声明环境变量
$ export TF_CLI_CONFIG_FILE=/home/terraform/youdianzhishi-terraform/terraform-alicloud-operator/.terraformrc
$ export ALICLOUD_ACCESS_KEY="xxxxxx"
$ export ALICLOUD_SECRET_KEY="xxxxxx"
$ export ALICLOUD_REGION="cn-shanghai"

# 测试执行计划
$ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  + create

Terraform will perform the following actions:

  # alicloud_oss_bucket.terra-backend will be created
  + resource "alicloud_oss_bucket" "terra-backend" {
      + acl               = "private"
      + bucket            = "terraform-backend-data-202209"
      + creation_date     = (known after apply)
      + extranet_endpoint = (known after apply)
      + force_destroy     = false
      + id                = (known after apply)
      + intranet_endpoint = (known after apply)
      + location          = (known after apply)
      + owner             = (known after apply)
      + redundancy_type   = "LRS"
      + storage_class     = "Standard"
    }

  # alicloud_ots_instance.terra-table will be created
  + resource "alicloud_ots_instance" "terra-table" {
      + accessed_by   = "Any"
      + description   = "terraform tablestore"
      + id            = (known after apply)
      + instance_type = "HighPerformance"
      + name          = "terra-table"
      + tags          = {
          + "Created" = "TF"
          + "For"     = "Building table"
        }
    }

  # alicloud_ots_table.basic will be created
  + resource "alicloud_ots_table" "basic" {
      + deviation_cell_version_in_sec = "1"
      + id                            = (known after apply)
      + instance_name                 = "terra-table"
      + max_version                   = 1
      + table_name                    = "terra_table"
      + time_to_live                  = -1

      + primary_key {
          + name = "LockID"
          + type = "String"
        }
    }

Plan: 3 to add, 0 to change, 0 to destroy.


# 创建资源
$ terraform apply -auto-approve

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

# 为backend添加`backend.tf`远程状态管理

# 进入backend项目目录下
$ cd youdianzhishi-terraform/terraform-alicloud-operator/global/backend

# 新建文件并编辑远程状态管理配置
$ vim backend.tf
terraform {
  backend "oss" {
    access_key          = "xxxxxx"
    secret_key          = "xxxxxx"
    bucket              = "terraform-backend-data-202209"
    prefix              = "backend/"
    # prefix              = "global/backend/"
    key                 = "terraform-backend.tfstate"
    region              = "cn-shanghai"
    tablestore_endpoint = "https://terra-table.cn-shanghai.ots.aliyuncs.com"
    tablestore_table    = "terra_table"
  }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# 三、申请专有网络资源

# VPC、Subnet资源申请

参考项目目录"youdianzhishi-terraform/terraform-alicloud-operator/env/dev/network"

这里的网络是固定的,不会被经常发生变更的

# 项目目录结构
$  tree ./
./
├── env
│   └── dev
├── global
│   └── backend
│       ├── backend.tf
│       ├── main.tf
│       ├── outputs.tf
│       ├── variables.tf
│       └── versions.tf
└── modules

5 directories, 7 files

# 进入`dev`目录,创建`network`项目目录
$ cd env/dev/
$ tree ./
./
├── backend.tf
├── main.tf
├── outputs.tf
├── variables.tf
└── versions.tf

0 directories, 5 files


$ terraform fmt
$ terraform validate
$ terraform plan
$ terraform apply -auto-approve

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

备注: 这里使用的都是terraform-alicloud-operator/modules集成的模块

# 申请安全组资源

参考上面的dev/network中添加安全组资源创建

# 进入`dev`目录结构
$ cd env/dev/
$ tree ./
./
├── backend.tf
├── main.tf
├── outputs.tf
├── variables.tf
└── versions.tf

0 directories, 5 files

# 添加secgroup资源创建段
$ vim main.tf
provider "alicloud" {
  access_key = var.alicloud_access_key
  secret_key = var.alicloud_secret_key
  region     = var.region
}

locals {
  ......
  vpc_id         = module.tf-vpc.vpc_id
}

....

module "devsecgoup" {
  source = "../../../modules/secgroup"
  vpc_id = local.vpc_id
}

$ terraform fmt
$ terraform validate
$ terraform plan
$ terraform apply -auto-approve

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

备注: 这里使用的都是terraform-alicloud-operator/modules集成的模块

# 四、申请ECS资源

参考项目目录"youdianzhishi-terraform/terraform-alicloud-operator"

这里在terraform-alicloud-operator/dev/目录下创建service项目目录

# 进入到service项目目录
$ cd terraform-alicloud-operator/env/dev/service

# `service`项目配置后的最终目录结构
tree  ./
./
├── backend.tf
├── ecs.tf
├── main.tf
├── outputs.tf
├── variables.tf
└── versions.tf

0 directories, 6 files


# 执行terraform命令
$ terraform fmt
$ terraform init 或 $ terraform init -plugin-dir $HOME/.terraform.d/terraform-plugin-cache
$ terraform validate
$ terraform plan
$ terraform apply 或 terraform apply -auto-approve

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# 报错处理

# 错误提示`因为在network没有把devsecgoup_id`给输出出来
╷
│ Error: Unsupported attribute
│ 
│   on ecs.tf line 6, in locals:
│    6:   secgroup_id                = data.terraform_remote_state.mydata.outputs.secgroup_id
│     ├────────────────
│     │ data.terraform_remote_state.mydata.outputs is object with 3 attributes
│ 
│ This object does not have an attribute named "secgroup_id".
╵


# 错误处理
$ cd env/dev/network/

# 声明变量
export ALICLOUD_ACCESS_KEY="xxxxxx"
export ALICLOUD_SECRET_KEY="xxxxxx"
export ALICLOUD_REGION="cn-shanghai"
export TF_CLI_CONFIG_FILE=/home/terraform/youdianzhishi-terraform/terraform-alicloud-operator/.terraformrc

# 再次执行apply
$ terraform apply
module.tf-vpc.alicloud_vpc.vpc: Refreshing state... [id=vpc-uf6mdg1ot5n6cqbtgya5r]
module.tf-vpc.alicloud_vswitch.vsw: Refreshing state... [id=vsw-uf6gw4kbxekl197tff0d6]
module.devsecgoup.alicloud_security_group.group: Refreshing state... [id=sg-uf6d43y7zolz1iesuoxc]
module.devsecgoup.alicloud_security_group_rule.allow_all_tcp: Refreshing state... [id=sg-uf6d43y7zolz1iesuoxc:egress:tcp:1/65535:intranet:0.0.0.0/0:accept:1]
module.devsecgoup.alicloud_security_group_rule.allow_22_tcp: Refreshing state... [id=sg-uf6d43y7zolz1iesuoxc:ingress:tcp:22/22:intranet:0.0.0.0/0:accept:1]
module.devsecgoup.alicloud_security_group_rule.allow_80_tcp: Refreshing state... [id=sg-uf6d43y7zolz1iesuoxc:ingress:tcp:80/80:intranet:0.0.0.0/0:accept:1]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

devsecgoup_id = "sg-uf6d43y7zolz1iesuoxc"  # 这里打印出安全组的ID信息
vpc_id = "vpc-uf6mdg1ot5n6cqbtgya5r"
vsw_id = "vsw-uf6gw4kbxekl197tff0d6"

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

# ECS运维初始化脚本

user_data: https://registry.terraform.io/providers/aliyun/alicloud/latest/docs/resources/instance

## 在`modules`目录下的`ecs`的main.tf中添加,nginx安装脚本内容
# 进入`terraform-alicloud-operator/modules/ecs`目录下
$ cd terraform-alicloud-operator/modules/ecs
$ vim main.tf

data "alicloud_images" "images_ds" {
......
}

resource "alicloud_instance" "myecs" {
......
  user_data = <<-EOF   # 在此处添加`user_data`的内容部分
              #!/bin/bash 
              # until [[ -f /var/lib/cloud/instance/boot-finished ]] ;do
              #    sleep 1
              # done

              yum -y install nginx 
              echo "dev nginx server!!!" >/usr/share/nginx/html/index.html
              systemctl restart nginx 
              EOF
......
}

# 声明变量
export ALICLOUD_ACCESS_KEY="xxxxxx"
export ALICLOUD_SECRET_KEY="xxxxxx"
export ALICLOUD_REGION="cn-shanghai"
export TF_CLI_CONFIG_FILE=/home/terraform/youdianzhishi-terraform/terraform-alicloud-operator/.terraformrc

# 然后进入到`terraform-alicloud-operator/env/dev/service`中
# 执行terraform命令
$ terraform fmt 或 terraform fmt -recursive
$ terraform init 或 terraform init -plugin-dir $HOME/.terraform.d/terraform-plugin-cache
$ terraform validate
$ terraform plan
$ terraform apply 或 terraform apply -auto-approve

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

验证nginx是否安装:

这里没有给ECS配置外网地址,因此需要从阿里云在浏览器上进行连接服务;然后验证nginx服务是否正常

# 申请SLB资源

https://registry.terraform.io/providers/aliyun/alicloud/latest/docs/resources/slb

这里使用的module方式来创建的slb(是传统网络型的负载均衡器),slb模块在terraform-alicloud-operator/modules/slb

#  进入`service`目录
$ cd terraform-alicloud-operator/env/dev/service
$ vim slb.tf  # 这里新建并编辑`slb.tf`文件
locals {
  address_type      = "intranet"
  slb_name          = "dev-slb"
  payment_type      = "PayAsYouGo"
  server_group_name = "webserver"
  # 传入所有的ecs实例
  ecs_ids            = module.dev-ecs[*].ecs_id
  backend_port       = 80
  backend_weight     = 100
  frontend_port      = 80
  protocol           = "http"
  scheduler          = "rr"
  load_balancer_spec = "slb.s1.small"
  bandwidth          = 10
  # 这里的`vsw_id`在`ecs.tf`中已经被使用了,所以需要进行调整
  lb_vsw_id = data.terraform_remote_state.mydata.outputs.vsw_id
}

module "dev-slb" {
  source             = "../../../modules/slb"
  address_type       = local.address_type
  slb_name           = local.slb_name
  payment_type       = local.payment_type
  server_group_name  = local.server_group_name
  ecs_ids            = local.ecs_ids
  backend_port       = local.backend_port
  backend_weight     = local.backend_weight
  frontend_port      = local.frontend_port
  protocol           = local.protocol
  scheduler          = local.scheduler
  lb_vsw_id          = local.lb_vsw_id
  load_balancer_spec = local.load_balancer_spec
  bandwidth          = local.bandwidth
}

# 声明变量
export ALICLOUD_ACCESS_KEY="xxxxxx"
export ALICLOUD_SECRET_KEY="xxxxxx"
export ALICLOUD_REGION="cn-shanghai"
export TF_CLI_CONFIG_FILE=/home/terraform/youdianzhishi-terraform/terraform-alicloud-operator/.terraformrc

# 执行terraform命令
$ terraform fmt 或 terraform fmt -recursive
$ terraform init 或 terraform init -plugin-dir $HOME/.terraform.d/terraform-plugin-cache
$ terraform validate
$ terraform plan
$ terraform apply 或 terraform apply -auto-approve

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

备注:

这里使用的都是terraform-alicloud-operator/modules集成的模块
进入ecs服务内部使用curl,验证负载均衡器是否能够正常提供服务

# 申请EIP和DND资源

# EIP

https://registry.terraform.io/providers/aliyun/alicloud/latest/docs/resources/eip

https://registry.terraform.io/providers/aliyun/alicloud/latest/docs/resources/eip_association

这里使用的module方式来创建的EIP,EIP模块在`terraform-alicloud-operator/modules/eip

#  进入`service`目录
$ cd terraform-alicloud-operator/env/dev/service
$ vim eip.tf  # 这里新建并编辑`eip.tf`文件
locals {
  instance_id = module.dev-slb.slb_id
}

# 将EIP关联与SLB_ID关联到一起
module "dev-eip" {
  source      = "../../../modules/eip"
  instance_id = local.instance_id
}

# 声明变量
export ALICLOUD_ACCESS_KEY="xxxxxx"
export ALICLOUD_SECRET_KEY="xxxxxx"
export ALICLOUD_REGION="cn-shanghai"
export TF_CLI_CONFIG_FILE=/home/terraform/youdianzhishi-terraform/terraform-alicloud-operator/.terraformrc

# 执行terraform命令
$ terraform fmt 或 terraform fmt -recursive
$ terraform init 或 terraform init -plugin-dir $HOME/.terraform.d/terraform-plugin-cache
$ terraform validate
$ terraform plan
$ terraform apply 或 terraform apply -auto-approve

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

备注:

这里使用的都是terraform-alicloud-operator/modules集成的模块
阿里云控制台验证SLB是否已经绑定好EIP
这里需要在阿里云RAM给用户添加EIP的操作权限

浏览器访问EIP地址验证后端服务是否正常

# DNS

https://registry.terraform.io/providers/aliyun/alicloud/latest/docs/resources/alidns_record

这里使用的module方式来创建的EIP,EIP模块在`terraform-alicloud-operator/modules/dns

#  进入`service`目录
$ cd terraform-alicloud-operator/env/dev/service
$ vim dns.tf  # 这里新建并编辑`dns.tf`文件
locals {
  dns_zone_name = "chsaos.com"
  dns_record    = "deva"
  eip           = module.dev-eip.eip
  record_type   = "A"
}

module "dev-dns" {
  source        = "../../../modules/dns"
  dns_zone_name = local.dns_zone_name
  eip           = local.eip
  record_type   = local.record_type
  dns_record    = local.dns_record

}

# 声明变量
export ALICLOUD_ACCESS_KEY="xxxxxx"
export ALICLOUD_SECRET_KEY="xxxxxx"
export ALICLOUD_REGION="cn-shanghai"
export TF_CLI_CONFIG_FILE=/home/terraform/youdianzhishi-terraform/terraform-alicloud-operator/.terraformrc

# 执行terraform命令
$ terraform fmt 或 terraform fmt -recursive
$ terraform init 或 terraform init -plugin-dir $HOME/.terraform.d/terraform-plugin-cache
$ terraform validate
$ terraform plan
$ terraform apply 或 terraform apply -auto-approve

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

备注:

这里使用的都是terraform-alicloud-operator/modules集成的模块
阿里云控制台查看DNS新增加的解析记录

域名直接访问测试效果

上次更新: 2024/04/09, 16:48:42

← Backend配置腾讯云实践→