mito’s blog

IT技術メインの雑記。思い立ったが吉日。

[Ansible 2.13] Terraformモジュールを使う際に気を付けるポイント

はじめに

AnsibleでTerraformモジュールを使った際に、気を付けるポイントを書き留めておきます。
以下にはまりました。

  1. Terraformのoutputの変数名にハイフンを使用していたため、Ansibleで呼び出せずエラーになった。
    • 原因はTerraformでは変数名にハイフンを使えるが、Ansibleでは使えないため。
  2. terraformモジュール実行後、次のタスクが失敗することがあった。
    • 原因はsshサービスが起動する前に次のタスクが走っていたため。
    • remote-execはインスタンスの起動待ちを気にしなくていいが、Playbookでは起動完了までwaitをいれたほうが安全。


解決策

以下で解決しました。

  1. outputの変数名をハイフンからアンダースコアに修正しました。
    • Terraform全体では、ハイフンとアンダースコアが混ざっている状態なのでちょっともやもや。
    • OSSを連携させるときに、こういった仕様も確認しなくては。
  2. shellモジュールを用いて、起動するまで待つステータスチェックのAWSコマンドを使用しました。
    • awsモジュールで実現したかったけど、見つけられませんでした。


環境

  • Ubuntu: 22.04.1
  • ansible core: 2.13.6
  • terraform: 1.3.4
  • community.general: 6.0.0
  • aws cli: 1.22.34


terraformで構築するインスタンス

outputの変数名はアンダースコアに修正済みです。
インスタンスIDはAWSコマンドの引数として使用するため、outputで出力しています。

provider "aws" {
    region = "ap-northeast-1"
    access_key = "XXXXX"
    secret_key = "XXXXX"
}

resource "aws_instance" "mito-ec2" {
    ami           = "ami-03f4fa076d2981b45"
    instance_type = "t2.micro"
    associate_public_ip_address = "true"
    vpc_security_group_ids = ["XXXXX"]
    key_name = "XXXXX"
    tags = {
        Name = "mito-ec2"
    }
}

output "ec2_id"{
    value = aws_instance.mito-ec2.id
}


実行するPlaybook

インスタンスの起動完了待ちは、aws ec2 wait instance-status-okコマンドを使用します。
引数に、outputで出力したインスタンスIDを指定します。

---
- hosts: localhost
  gather_facts: no

  tasks:
  - name: terraform apply
    community.general.terraform:
      project_path: 'terraform/'
      force_init: yes
      state: present
    register: result

  - name: wait instance_status_ok
    shell:
      cmd: "aws ec2 wait instance-status-ok --instance-ids {{ result.outputs.ec2_id.value }}"

  - name: get instance_status
    shell:
      cmd: "aws ec2 describe-instance-status --instance-ids {{ result.outputs.ec2_id.value }}"
    register: result_instance_status

  - name: debug
    debug:
      msg: "{{ result_instance_status.stdout_lines }}"


実行ログ

Playbookを実行します。
aws ec2 describe-instance-statusの実行結果で、起動完了が確認できます。

$ ansible-playbook terraform_module.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the 
implicit localhost does not match 'all'

PLAY [localhost] **************************************************************

TASK [terraform apply] ********************************************************
changed: [localhost]

TASK [wait instance_status_ok] ************************************************
changed: [localhost]

TASK [get instance_status] ****************************************************
changed: [localhost]

TASK [debug] *****************************************************************
ok: [localhost] => {
    "msg": [
        "{",
        "    "InstanceStatuses": [",
        "        {",
        "            "AvailabilityZone": "ap-northeast-1c",",
        "            "InstanceId": "i-0080415c81f9dd541",",
        "            "InstanceState": {",
        "                "Code": 16,",
        "                "Name": "running"",
        "            },",
        "            "InstanceStatus": {",
        "                "Details": [",
        "                    {",
        "                        "Name": "reachability",",
        "                        "Status": "passed"",
        "                    }",
        "                ],",
        "                "Status": "ok"",
        "            },",
        "            "SystemStatus": {",
        "                "Details": [",
        "                    {",
        "                        "Name": "reachability",",
        "                        "Status": "passed"",
        "                    }",
        "                ],",
        "                "Status": "ok"",
        "            }",
        "        }",
        "    ]",
        "}"
    ]
}

PLAY RECAP *******************************************************************
localhost: ok=4 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0   

$ 


エラーメッセージ

outputの変数にハイフンを使った場合のエラーで、ec2-idとしています。
リソースは作成されますが、waitで失敗します。

$ ansible-playbook terraform_module.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the 
implicit localhost does not match 'all'

PLAY [localhost] ***************************************************************

TASK [terraform apply] *********************************************************
changed: [localhost]

TASK [wait instance_status_ok] *************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined 
variable. The error was: 'dict object' has no attribute 'ec2'\n\nThe error appears to 
be in '/home/ubuntu/test/terraform_module.yml': line 13, column 5, but may\nbe 
elsewhere in the file depending on the exact syntax problem.\n\nThe offending line 
appears to be:\n\n\n  - name: wait instance_status_ok\n    ^ here\n"}

PLAY RECAP *********************************************************************
localhost: ok=1 changed=1 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0   

$ 


[Ansible] Terraformモジュールを使って、terraform applyを実行する

はじめに

AnsibleとTerraformを組み合わせると楽できそうだったので、AnsibleのTerraformモジュールを使ってみました。

community.general.terraform module – Manages a Terraform deployment (and plans) — Ansible Documentation

terraformとansibleを機能と用途から比較してみた、今の感想です。

  • Terraformはtfstateもあり作ったリソース(ガワ)を管理しやすいが、中身の構築はremote-exec(Ansibleのshellモジュール相当)しかない。
  • Ansibleはモジュールが冪等性を持っているので中身を構築しやすいが、リソース(ガワ)の管理は外部に任すことになる。


環境

  • Ubuntu: 22.04.1
  • ansible core: 2.13.6
  • terraform: 1.3.4
  • community.general: 6.0.0


tfファイル

tfファイルは、ubuntuを1台立てます。
outputでパブリックIPを表示させ、どのような感じでログが出力されるかも確認します。

provider "aws" {
    region = "ap-northeast-1"
    access_key = "XXXXXX"
    secret_key = "XXXXXX"
}

resource "aws_instance" "mito-ec2" {
    ami           = "ami-03f4fa076d2981b45"
    instance_type = "t2.micro"
    associate_public_ip_address = "true"
    tags = {
        Name = "mito-ec2"
    }
}

output "ec2_ip"{
    value = aws_instance.mito-ec2.public_ip
}


terraformモジュールを使ったPlaybook

terraform apply相当のPlaybookです。
ログを確認したいため、実行結果をデバッグ表示します。

---
- hosts: localhost
  gather_facts: no

  tasks:
  - name: terraform apply
    community.general.terraform:
      project_path: 'terraform/'   # Terraform ディレクトリのルートへのパス
      force_init: yes      # terraform initの実行
      state: present
    register: result
  
  - name: debug
    debug:
      msg: "{{ result }}"


terraform apply相当のPlaybook実行

Playbook実行前のディレクトリ構造です。
terraformのプラグインはまだダウンロードしていません。

$ tree
.
├── terraform
│   └── main.tf
└── terraform_module.yml

1 directory, 2 files
$ 



terraform apply相当のPlaybookを実行します。

結果、outputsのパブリックIPが無事表示されました。これなら扱いやすいです。
terraformコマンドのようにApply complete!も表示されました。

$ ansible-playbook terraform_module.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ****************************************************************************************

TASK [terraform apply] **********************************************************************************
changed: [localhost]

TASK [debug] ********************************************************************************************
ok: [localhost] => {
    "msg": {
        "changed": true,
        "command": "/usr/bin/terraform apply -no-color -input=false -auto-approve -lock=true /tmp/tmp8yoeiy_e.tfplan",
        "failed": false,
        "outputs": {
            "ec2_ip": {
                "sensitive": false,
                "type": "string",
                "value": "3.113.21.61"
            }
        },
        "state": "present",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "aws_instance.mito-ec2: Creating...aws_instance.mito-ec2: Still creating... [10s 
        elapsed]aws_instance.mito-ec2: Still creating... [20s elapsed]aws_instance.mito-ec2: Creation 
        complete after 22s [id=i-02d5e9a2f24dc2de8]Apply complete! Resources: 1 added, 0 changed, 0 
        destroyed.Outputs:ec2_ip = \"3.113.21.61\"",
        "stdout_lines": [
            "aws_instance.mito-ec2: Creating...",
            "aws_instance.mito-ec2: Still creating... [10s elapsed]",
            "aws_instance.mito-ec2: Still creating... [20s elapsed]",
            "aws_instance.mito-ec2: Creation complete after 22s [id=i-02d5e9a2f24dc2de8]",
            "",
            "Apply complete! Resources: 1 added, 0 changed, 0 destroyed.",
            "",
            "Outputs:",
            "",
            "ec2_ip= \"3.113.21.61\""
        ],
        "workspace": "default"
    }
}

PLAY RECAP **********************************************************************************************
localhost      : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

$ 


ディレクトリ構造をみると、プラグインもダウンロードされています。

$ tree -a
.
├── terraform
│   ├── .terraform
│   │   └── providers
│   │       └── registry.terraform.io
│   │           └── hashicorp
│   │               └── aws
│   │                   └── 4.39.0
│   │                       └── linux_amd64
│   │                           └── terraform-provider-aws_v4.39.0_x5
│   ├── .terraform.lock.hcl
│   ├── main.tf
│   └── terraform.tfstate
└── terraform_module.yml

8 directories, 5 files
$ 


terraform apply相当のPlaybook実行(2回目)

Playbookを再実行してみます。

Taskの実行結果はokで、ログNo changesも表示されました。
問題なさそうです。

$ ansible-playbook terraform_module.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ****************************************************************************************

TASK [terraform apply] **********************************************************************************
ok: [localhost]

TASK [debug] ********************************************************************************************
ok: [localhost] => {
    "msg": {
        "changed": false,
        "command": "/usr/bin/terraform apply -no-color -input=false -auto-approve -lock=true /tmp/
        tmprz34gv4k.tfplan",
        "failed": false,
        "outputs": {
            "ec2_ip": {
                "sensitive": false,
                "type": "string",
                "value": "3.113.21.61"
            }
        },
        "state": "present",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "aws_instance.mito-ec2: Refreshing state... [id=i-02d5e9a2f24dc2de8]No changes. Your 
        infrastructure matches the configuration.Terraform has compared your real infrastructure against 
        your configurationand found no differences, so no changes are needed.",
        "stdout_lines": [
            "aws_instance.mito-ec2: Refreshing state... [id=i-02d5e9a2f24dc2de8]",
            "",
            "No changes. Your infrastructure matches the configuration.",
            "",
            "Terraform has compared your real infrastructure against your configuration",
            "and found no differences, so no changes are needed."
        ],
        "workspace": "default"
    }
}

PLAY RECAP *********************************************************************************************
localhost      : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

$ 


terraform destroy相当のPlaybook実行

Playbookのstateを、presentからabsentに書き換えて実行します。

結果、applyと同様に、terraformコマンドの結果が表示され、Destroy complete!も確認できました。
plan結果も出力されています。

$ ansible-playbook terraform_module.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ****************************************************************************************

TASK [terraform destroy] ********************************************************************************
changed: [localhost]

TASK [debug] ********************************************************************************************
ok: [localhost] => {
    "msg": {
        "changed": true,
        "command": "/usr/bin/terraform destroy -no-color -auto-approve -lock=true",
        "failed": false,
        "outputs": {},
        "state": "absent",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "aws_instance.mito-ec2: Refreshing state... [id=i-02d5e9a2f24dc2de8] Terraform used the 
        selected providers to generate the following executionplan. Resource actions are indicated with 
        the following symbols:  - destroy  Terraform will perform the following actions:  # aws_instance.
        mito-ec2 will be destroyed  - resource \"aws_instance\" \"mito-ec2\" {      - 
        ami                                  = \"ami-03f4fa076d2981b45\" -> null      - 
        (略)
        Plan: 0 to add, 0 to change, 1 to destroy.
        Changes to Outputs:  - ec2_ip = \"3.113.21.61\" -> null aws_instance.mito-ec2: 
        Destroying... [id=i-02d5e9a2f24dc2de8] aws_instance.mito-ec2: Still destroying... 
        [id=i-02d5e9a2f24dc2de8, 10s elapsed] aws_instance.mito-ec2: Still destroying... 
        [id=i-02d5e9a2f24dc2de8, 20s elapsed] aws_instance.mito-ec2: Still destroying... 
        [id=i-02d5e9a2f24dc2de8, 30s elapsed] aws_instance.mito-ec2: Destruction complete after 
        40s Destroy complete! Resources: 1 destroyed.",
        "stdout_lines": [
            "aws_instance.mito-ec2: Refreshing state... [id=i-02d5e9a2f24dc2de8]",
            "",
            "Terraform used the selected providers to generate the following execution",
            "plan. Resource actions are indicated with the following symbols:",
            "  - destroy",
            "",
            "Terraform will perform the following actions:",
            "",
            "  # aws_instance.mito-ec2 will be destroyed",
            "  - resource \"aws_instance\" \"mito-ec2\" {",
            "      - ami                                  = \"ami-03f4fa076d2981b45\" -> null",
            (略)
            "      - root_block_device {",
            "          - delete_on_termination = true -> null",
            "          - device_name           = \"/dev/sda1\" -> null",
            "          - encrypted             = false -> null",
            "          - iops                  = 100 -> null",
            "          - tags                  = {} -> null",
            "          - throughput            = 0 -> null",
            "          - volume_id             = \"vol-0a085b087b420fd22\" -> null",
            "          - volume_size           = 8 -> null",
            "          - volume_type           = \"gp2\" -> null",
            "        }",
            "    }",
            "",
            "Plan: 0 to add, 0 to change, 1 to destroy.",
            "",
            "Changes to Outputs:",
            "  - ec2_ip = \"3.113.21.61\" -> null",
            "aws_instance.mito-ec2: Destroying... [id=i-02d5e9a2f24dc2de8]",
            "aws_instance.mito-ec2: Still destroying... [id=i-02d5e9a2f24dc2de8, 10s elapsed]",
            "aws_instance.mito-ec2: Still destroying... [id=i-02d5e9a2f24dc2de8, 20s elapsed]",
            "aws_instance.mito-ec2: Still destroying... [id=i-02d5e9a2f24dc2de8, 30s elapsed]",
            "aws_instance.mito-ec2: Destruction complete after 40s",
            "",
            "Destroy complete! Resources: 1 destroyed."
        ],
        "workspace": "default"
    }
}

PLAY RECAP **********************************************************************************************
localhost      : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

$ 


初回実行時にinitしていない場合のエラー

Playbookの初回実行時にforce_init: yesがない場合、プラグインがないため、以下のエラーメッセージが出力されます。

$ ansible-playbook terraform_module.yml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ****************************************************************************************

TASK [terraform apply] **********************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/usr/bin/terraform validate", "msg": "╷Error: 
Missing required providerThis configuration requires provider registry.terraform.io/hashicorp/aws,but 
that provider isn't available. You may be able to install itautomatically by running:  terraform init╵", 
"rc": 1, "stderr": "╷Error: Missing required providerThis configuration requires provider registry.
terraform.io/hashicorp/aws,but that provider isn't available. You may be able to install itautomatically 
by running:  terraform init╵", "stderr_lines": ["", "Error: Missing required provider", "", "This 
configuration requires provider registry.terraform.io/hashicorp/aws,", "but that provider isn't 
available. You may be able to install it", "automatically by running:", "  terraform init", ""], 
"stdout": "", "stdout_lines": []}

PLAY RECAP **********************************************************************************************
localhost      : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

$ 


備考

Terraformモジュールは、デフォルト設定のパラメータがいくつかあります。
一部のデフォルト値と内容をメモします。

  • force_init: no
    • yesの場合、terraform initを実行する
  • check_destroy: no
    • yesの場合、applyによりdestroyのみが発生する場合は実行しない。ただしdestroyしてaddする場合は実行する
    • yesでも、stateがabsentの場合(terraform destroy)は実行する
  • overwrite_init : yes
    • project_pathに、tfstateが既に存在する場合でも init を実行する

[Terraform]モジュールで定義した変数を参照する

はじめに

モジュールで定義した変数を参照するには、outputで定義する必要があります。

先日同僚から教わった、分かりやすいたとえがこちらです。

モジュールはそれ単体が一つの独立したTerraform、あるいはアプリケーションだと思っていただくとイメージしやすいです。 通常、独立したアプリケーションの実行結果を受けとろうと思ったら、APIを使用すると思います。 このAPIの結果を返す側のインターフェースが「output」です。


公式ドキュメント
Output Values - Configuration Language | Terraform by HashiCorp


コード例

モジュールで取得したEC2インスタンスのタグ名を参照します。

$ cat main.tf 
provider "aws" {
  region = "ap-northeast-1"
}

module "instance_name"{
  source   = "./modules/instance_name"
}

output "sample"{
  value = module.instance_name
}
$ 
$ cat modules/instance_name/main.tf 
data "aws_instance" "mito" {
  filter {
    name   = "tag:Name"
    values = ["mito-*"]
  }
}

output "tag_name" {
  value = data.aws_instance.mito.tags
}
$ 


各コードの実行例

上記コードをapplyします

モジュールで定義した変数が参照できました。

$ terraform apply

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

sample = {
  "tag_name" = tomap({
    "Name" = "mito-ec2-X"
  })
}
$ 


直接モジュールの変数を参照し、applyします

output "sample"のvalueに、モジュールの変数を指定します。
結果、定義されていない変数への参照となり、エラーになりました。

$ cat main.tf 
provider "aws" {
  region = "ap-northeast-1"
}

module "instance_name"{
  source   = "./modules/instance_name"
}

output "sample"{
  // モジュールの変数を直接指定
  value = data.aws_instance.mito.tags
}
$ 
$ cat modules/instance_name/main.tf 
data "aws_instance" "mito" {
  filter {
    name   = "tag:Name"
    values = ["mito-*"]
  }
}

output "tag_name" {
  value = data.aws_instance.mito.tags
}
$ 
$ terraform apply
╷
│ Error: Reference to undeclared resource
│ 
│   on main.tf line 10, in output "sample":
│   10:   value = data.aws_instance.mito.tags
│ 
│ A data resource "aws_instance" "mito" has not been declared in the root module.
╵
$ 


モジュールのoutputを削除し、applyします

モジュールのoutput "tag_name"をコメントアウトしました。
結果、空のデータが表示されました。

$ cat main.tf 
provider "aws" {
  region = "ap-northeast-1"
}

module "instance_name"{
  source   = "./modules/instance_name"
}

output "sample"{
  value = module.instance_name
}
$ 
$ cat modules/instance_name/main.tf 
data "aws_instance" "mito" {
  filter {
    name   = "tag:Name"
    values = ["mito-*"]
  }
}

/* コメントアウトします
output "tag_name" {
  value = data.aws_instance.mito.tags
}
*/
$ 
$ terraform apply

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

sample = {}
$  

[Snyk][Terraform] Snykで実リソースとtfstateファイルの差分を検出してみた

はじめに

実リソースとコードの差異を検出することをドリフト検出と言うそうです。

今回は、Terraformで作成したAWSリソースとtfstateファイルの差異を検出します。
Snykで試したかったことがこれです!


まとめ

ドリフト検出を試して、おおむね満足のいく結果になりました。SGのインバウンドルール追加は検出されませんでしたが。理由は調査中です。
使いこなすには、ポリシーで何を検出しないか決めたり、コードの定義もデフォルトに任せず明記したほうがいいかも。
あと、結果をJSON形式で出力すれば、tfstateファイルに管理外のリソースをインポートしやすくなるかもしれないですね。


目次


環境

  • Cloud9: t3.small
  • Snyk CLI: 1.992.0
  • Terraform: 1.2.8


試すこと

TerraformでEC2インスタンスを構築した後、手動でEIPとSGのインバウンドルールを追加し、ドリフト検出します。
実行コマンドは以下です。

$ snyk iac describe
IaC describe
Usage
  Note: This feature is available in Snyk CLI version v1.876.0 or greater.

  snyk iac describe [<OPTIONS>]

Description
  The snyk iac describe command detects infrastructure drift and unmanaged resources. It compares
  resources in your Terraform state file against actual resources in your cloud provider and outputs
  a report.

  -  Resources in your Terraform state files are managed resources.
  -  Changes to managed resources not reflected in the Terraform state file are drifts.
  -  Resources that exist but are not in your Terraform state file are unmanaged resources.

  For detailed information and examples, see IaC describe command examples https://docs.snyk.io/prod
  ucts/snyk-infrastructure-as-code/detect-drift-and-manually-created-resources/iac-describe-command-
  examples

  For a list of related commands see the snyk iac help; iac --help

Exit codes
  Possible exit codes and their meaning:

  0: success, no drift found
  1: drifts or unmanaged resources found
  2: failure

Required options
  Note: To use the describe command, you must use one of these options:

  --only-unmanaged
    Report resources not found in any Terraform states.

  --only-managed or --drift
    Scan managed resources found in Terraform states for changes.

  --all
    Scan both managed and unmanaged resources.


リソース作成前にドリフト検出する

tfstateファイルは存在しないので、これといった結果はありません。
「IaC Coverage: 0%」が重要そうですね。

$ snyk iac describe --drift
Scanned states (1) 
Scan duration: 16s     
Provider version used to scan: 4.28.0. Use --tf-provider-version to use another version.
Snyk Scanning Infrastructure As Code Discrepancies...

  Info:    Resources under IaC, but different to terraform states.
  Resolve: Reapply IaC resources or update into terraform.

Test Summary


  IaC Coverage: 0%
  Info: To reach full coverage, remove resources or move it to Terraform.

  Tip: Run --help to find out about commands and flags.
      Scanned with aws provider version 4.28.0. Use --tf-provider-version to update.


TerraformでEC2インスタンスを構築する

ドリフト検出するために、以下のtfファイルでEC2インスタンスを構築します。

provider "aws" {
    region        = "ap-northeast-1"
}

resource "aws_instance" "mito-ec2" {
    ami           = "ami-0f36dcfcc94112ea1"
    instance_type = "t2.micro"
    key_name      = "キー名"
    associate_public_ip_address = "true"
    
    vpc_security_group_ids = [
        aws_security_group.mito_sg.id
    ]
  
    tags = {
        Name      = "mito-ec2"
    }
}

resource "aws_security_group" "mito_sg" {
  name            = "mito_sg"

  # インバウンド
  dynamic "ingress" {
    for_each      = [22, 80]

    content {
      from_port   = ingress.value
      to_port     = ingress.value
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
    }
  }

  # アウトバウンド
  egress {
    from_port     = 0
    to_port       = 0
    protocol      = "-1"
    cidr_blocks   = ["0.0.0.0/0"]
  }
}


EC2インスタンスを変更する前に、ドリフト検出する

Changed Resourcesが2となっているけど、まぁ今回は気にしない。
Ignore resourcesで、検出したくないリソースが指定できそうです。

$ snyk iac describe --drift
Scanned states (1) 
Scan duration: 14s     
Provider version used to scan: 4.28.0. Use --tf-provider-version to use another version.
Snyk Scanning Infrastructure As Code Discrepancies...

  Info:    Resources under IaC, but different to terraform states.
  Resolve: Reapply IaC resources or update into terraform.

Changed resources: 2

State: Generated [ Changed Resources: 1 ]

  Resource Type: aws_ebs_volume
    ID: vol-056e51aa5a6148297
    + timeouts: null => {}

State: tfstate://terraform.tfstate [ Changed Resources: 1 ]

  Resource Type: aws_instance
    ID: i-0683a97d236667e6f
    ~ arn: arn:aws:ec2:ap-northeast-1:xxxx:instance/i-0683a97d236667e6f => arn:aws:ec2:ap-northeast-1::instance/i-0683a97d236667e6f
    - user_data_replace_on_change: false

Test Summary

  Managed Resources: 3
  Changed Resources: 2

  IaC Coverage: 100%
  Info: To reach full coverage, remove resources or move it to Terraform.

  Tip: Run --help to find out about commands and flags.
      Scanned with aws provider version 4.28.0. Use --tf-provider-version to update.


EIPを付与した後に、ドリフト検出する

手動でEC2インスタンスに付与し、ドリフト検出してみました。
結果、「public_dns」と「public_ip」が検出されました。想定通りです。

$ snyk iac describe --drift
Scanned states (1) 
Scan duration: 7s      
Provider version used to scan: 4.28.0. Use --tf-provider-version to use another version.
Snyk Scanning Infrastructure As Code Discrepancies...

  Info:    Resources under IaC, but different to terraform states.
  Resolve: Reapply IaC resources or update into terraform.

Changed resources: 2

State: Generated [ Changed Resources: 1 ]

  Resource Type: aws_ebs_volume
    ID: vol-056e51aa5a6148297
    + timeouts: null => {}

State: tfstate://terraform.tfstate [ Changed Resources: 1 ]

  Resource Type: aws_instance
    ID: i-0683a97d236667e6f
    ~ arn: arn:aws:ec2:ap-northeast-1:xxxx:instance/i-0683a97d236667e6f => arn:aws:ec2:ap-northeast-1::instance/i-0683a97d236667e6f
    ~ public_dns: ec2-54-65-55-184.ap-northeast-1.compute.amazonaws.com => ec2-52-69-152-84.ap-northeast-1.compute.amazonaws.com
    ~ public_ip: 54.65.55.184 => 52.69.152.84
    - user_data_replace_on_change: false

Test Summary

  Managed Resources: 3
  Changed Resources: 2

  IaC Coverage: 100%
  Info: To reach full coverage, remove resources or move it to Terraform.

  Tip: Run --help to find out about commands and flags.
      Scanned with aws provider version 4.28.0. Use --tf-provider-version to update.


SGのインバウンドルールを追加した後に、ドリフト検出する

インバウンドルールに443を追加しましたが、これは検出されませんでした。

$ snyk iac describe --drift
Scanned states (1) 
Scan duration: 7s      
Provider version used to scan: 4.28.0. Use --tf-provider-version to use another version.
Snyk Scanning Infrastructure As Code Discrepancies...

  Info:    Resources under IaC, but different to terraform states.
  Resolve: Reapply IaC resources or update into terraform.

Changed resources: 2

State: Generated [ Changed Resources: 1 ]

  Resource Type: aws_ebs_volume
    ID: vol-056e51aa5a6148297
    + timeouts: null => {}

State: tfstate://terraform.tfstate [ Changed Resources: 1 ]

  Resource Type: aws_instance
    ID: i-0683a97d236667e6f
    ~ arn: arn:aws:ec2:ap-northeast-1:xxxx:instance/i-0683a97d236667e6f => arn:aws:ec2:ap-northeast-1::instance/i-0683a97d236667e6f
    ~ public_dns: ec2-54-65-55-184.ap-northeast-1.compute.amazonaws.com => ec2-52-69-152-84.ap-northeast-1.compute.amazonaws.com
    ~ public_ip: 54.65.55.184 => 52.69.152.84
    - user_data_replace_on_change: false

Test Summary

  Managed Resources: 3
  Changed Resources: 2

  IaC Coverage: 100%
  Info: To reach full coverage, remove resources or move it to Terraform.

  Tip: Run --help to find out about commands and flags.
      Scanned with aws provider version 4.28.0. Use --tf-provider-version to update.


EC2インスタンスを削除して、ドリフト検出する

EC2インスタンスを削除し、SGは残したままでドリフト検出しました。
結果、Missing Resourceとaws_instanceは、Missing Resourcesとして検出されました。 残ったManaged ResourcesはSGのようです。

$ snyk iac describe --drift
Scanned states (1) 
Scan duration: 8s      
Provider version used to scan: 4.28.0. Use --tf-provider-version to use another version.
Snyk Scanning Infrastructure As Code Discrepancies...

  Info:    Resources under IaC, but different to terraform states.
  Resolve: Reapply IaC resources or update into terraform.

Missing resources: 2

State: Generated [ Missing Resources: 1 ]

  Resource Type: aws_ebs_volume
    ID: vol-056e51aa5a6148297

State: tfstate://terraform.tfstate [ Missing Resources: 1 ]

  Resource Type: aws_instance
    ID: i-0683a97d236667e6f

Test Summary

  Managed Resources: 1
  Missing Resources: 2

  IaC Coverage: 33%
  Info: To reach full coverage, remove resources or move it to Terraform.

  Tip: Run --help to find out about commands and flags.
      Scanned with aws provider version 4.28.0. Use --tf-provider-version to update.


JSON形式で、ドリフト検出の結果を出力する

JSON形式で出力した場合、出管理下にあるリソース、管理下にないリソースが分かりやすいですね。

$ snyk iac describe --json --drift
Scanned states (1) 
Scan duration: 7s      
Provider version used to scan: 4.28.0. Use --tf-provider-version to use another version.
{
        "options": {
                "deep": true,
                "only_managed": true,
                "only_unmanaged": false
        },
        "summary": {
                "total_resources": 3,
                "total_changed": 2,
                "total_unmanaged": 0,
                "total_missing": 0,
                "total_managed": 3,
                "total_iac_source_count": 1
        },
        "managed": [
                {
                        "id": "vol-02205209f17fcac8b",
                        "type": "aws_ebs_volume"
                },
                {
                        "id": "i-01b5dbaa8fadcd935",
                        "type": "aws_instance",
                        "human_readable_attributes": {
                                "Name": "mito-ec2"
                        },
                        "source": {
                                "source": "tfstate://terraform.tfstate",
                                "namespace": "",
                                "internal_name": "mito-ec2"
                        }
                },
                {
                        "id": "sg-0354feb133037f2f0",
                        "type": "aws_security_group",
                        "source": {
                                "source": "tfstate://terraform.tfstate",
                                "namespace": "",
                                "internal_name": "mito_sg"
                        }
                }
        ],
        "unmanaged": null,
        "missing": null,
        "differences": [
                {
                        "res": {
                                "id": "vol-02205209f17fcac8b",
                                "type": "aws_ebs_volume"
                        },
                        "changelog": [
                                {
                                        "type": "create",
                                        "path": [
                                                "timeouts"
                                        ],
                                        "from": null,
                                        "to": {},
                                        "computed": false
                                }
                        ]
                },
                {
                        "res": {
                                "id": "i-01b5dbaa8fadcd935",
                                "type": "aws_instance",
                                "human_readable_attributes": {
                                        "Name": "mito-ec2"
                                },
                                "source": {
                                        "source": "tfstate://terraform.tfstate",
                                        "namespace": "",
                                        "internal_name": "mito-ec2"
                                }
                        },
                        "changelog": [
                                {
                                        "type": "update",
                                        "path": [
                                                "arn"
                                        ],
                                        "from": "arn:aws:ec2:ap-northeast-1:xxxx:instance/i-01b5dbaa8fadcd935",
                                        "to": "arn:aws:ec2:ap-northeast-1::instance/i-01b5dbaa8fadcd935",
                                        "computed": true
                                },
                                {
                                        "type": "update",
                                        "path": [
                                                "public_dns"
                                        ],
                                        "from": "ec2-35-77-229-240.ap-northeast-1.compute.amazonaws.com",
                                        "to": "ec2-54-65-228-246.ap-northeast-1.compute.amazonaws.com",
                                        "computed": true
                                },
                                {
                                        "type": "update",
                                        "path": [
                                                "public_ip"
                                        ],
                                        "from": "35.77.229.240",
                                        "to": "54.65.228.246",
                                        "computed": true
                                },
                                {
                                        "type": "delete",
                                        "path": [
                                                "user_data_replace_on_change"
                                        ],
                                        "from": false,
                                        "to": null,
                                        "computed": false
                                }
                        ]
                }
        ],
        "coverage": 100,
        "alerts": {
                "": [
                        {
                                "message": "You have unmanaged security group rules that could be false positives, find out more at https://docs.driftctl.com/limitations"
                        },
                        {
                                "message": "You have diffs on computed fields, check the documentation for potential false positive drifts: https://docs.driftctl.com/limitations"
                        }
                ]
        },
        "provider_name": "aws",
        "provider_version": "4.28.0",
        "scan_duration": 7,
        "date": "2022-08-29T08:11:36.782188621Z"
}


オプション「--all」による検出

全リソースが11sで検出されました。早いですね。

$ snyk iac describe --all
Scanned states (1) 
Scan duration: 11s     
Provider version used to scan: 4.28.0. Use --tf-provider-version to use another version.
Snyk Scanning Infrastructure As Code Discrepancies...

  Info:    Resources under IaC, but different to terraform states.
  Resolve: Reapply IaC resources or update into terraform.

Changed resources: 2

State: Generated [ Changed Resources: 1 ]

  Resource Type: aws_ebs_volume
    ID: vol-02205209f17fcac8b
    + timeouts: null => {}

State: tfstate://terraform.tfstate [ Changed Resources: 1 ]

  Resource Type: aws_instance
    ID: i-01b5dbaa8fadcd935
    ~ arn: arn:aws:ec2:ap-northeast-1:xxxx:instance/i-01b5dbaa8fadcd935 => arn:aws:ec2:ap-northeast-1::instance/i-01b5dbaa8fadcd935
    ~ public_dns: ec2-35-77-229-240.ap-northeast-1.compute.amazonaws.com => ec2-54-65-228-246.ap-northeast-1.compute.amazonaws.com
    ~ public_ip: 35.77.229.240 => 54.65.228.246
    - user_data_replace_on_change: false

Unmanaged resources: 71

Service: Unidentified [ Unmanaged Resources: 8 ]

  Resource Type: aws_cloudformation_stack
    ID: xxxx

  Resource Type: aws_iam_group
    ID: xxxx

  Resource Type: aws_iam_group_policy
    ID: xxxx

  Resource Type: aws_network_acl_rule
    ID: xxxx

  Resource Type: aws_s3_bucket_public_access_block
    ID: xxxx

Service: aws_ec2 [ Unmanaged Resources: 13 ]

  Resource Type: aws_ebs_snapshot
    ID: xxxx

  Resource Type: aws_ebs_volume
    ID: xxxx
  
  Resource Type: aws_eip
    ID: xxxx

  Resource Type: aws_eip_association
    ID: xxxx

  Resource Type: aws_instance
    ID: xxxx

  Resource Type: aws_key_pair
    ID: xxxx

Service: aws_iam [ Unmanaged Resources: 26 ]

  Resource Type: aws_iam_access_key
    ID: xxxx

  Resource Type: aws_iam_policy
    ID: xxxx

  Resource Type: aws_iam_policy_attachment
    ID: xxxx

  Resource Type: aws_iam_role
    ID: AWSCloud9SSMAccessRole

  Resource Type: aws_iam_role_policy
    ID: xxxx

  Resource Type: aws_iam_user
    ID: xxxx

Service: aws_route53 [ Unmanaged Resources: 1 ]

  Resource Type: aws_route53_zone
    ID: xxxxx

Service: aws_s3 [ Unmanaged Resources: 1 ]

  Resource Type: aws_s3_bucket
    ID: cf-templates-xxxx-ap-northeast-1

Service: aws_vpc [ Unmanaged Resources: 22 ]

  Resource Type: aws_security_group
    ID: sg-xxxxx

  Resource Type: aws_security_group_rule
    ID: sgrule-xxxx

Test Summary

  Managed Resources: 3
  Changed Resources: 2
  Unmanaged Resources: 71

  IaC Coverage: 4%
  Info: To reach full coverage, remove resources or move it to Terraform.

  Tip: Run --help to find out about commands and flags.
      Scanned with aws p


参考

Snyk IaCでTerraformのドリフト検出をやってみた | DevelopersIO

[Snyk] Snyk CLIをインストールする

はじめに

先日開催されたCNSC2022やHashiTalks:Japanで紹介されたSnykが気になったので、触ってみました。
本記事ではDockerイメージのスキャンを試していますが、こちらの記事では、Snykでドリフト検出(実リソースとtfstateの差異を検出)を試しています。


環境


Snykのアカウント登録

SnykのWebサイトを開き、アカウントを登録します。

なお、無料アカウントは以下の制限があります。


Snyk CLIのインストール

手順に沿って、Snyk CLIをインストールします。

Install or update the Snyk CLI - Snyk User Docs

$ curl https://static.snyk.io/cli/latest/snyk-linux -o snyk
$ chmod +x ./snyk
$ sudo mv ./snyk /usr/local/bin/


Snykのアカウント認証

APIトークンを払い出し、認証します。

  1. WebページのAccount Settingsに移動します。
  2. General->Auth TokenのKEYをコピーします。
  3. CLIでsnyk auth <API_TOKEN>を実行します。

Authenticate the CLI with your account - Snyk User Docs

$ snyk auth APIトークン
Your account has been authenticated. Snyk is now ready to be used.


Snyk CLIでDockerイメージをスキャンする

hello-worldをスキャンしてみます。

$ snyk container test hello-world

Testing hello-world...

Organization:      mito-201
Package manager:   linux
Project name:      docker-image|hello-world
Docker image:      hello-world
Platform:          linux/amd64
Licenses:          enabled

✔ Tested hello-world for known issues, no vulnerable paths found.

Note that we do not currently have vulnerability data for your image. 


Snyk CLIのヘルプを表示

色々できそうですね。

$ snyk --help
CLI commands help
  Snyk CLI scans and monitors your projects for security vulnerabilities and license issues.

  For more information visit the Snyk website https://snyk.io

  For details see the CLI documentation https://docs.snyk.io/features/snyk-cli

How to get started
  1. Authenticate by running snyk auth
  2. Test your local project with snyk test
  3. Get alerted for new vulnerabilities with snyk monitor

Available commands
  To learn more about each Snyk CLI command, use the --help option, for example, snyk auth --help or 
  snyk container --help

  snyk auth
    Authenticate Snyk CLI with a Snyk account.

  snyk test
    Test a project for open source vulnerabilities and license issues.

    Note: Use snyk test --unmanaged to scan all files for known open source dependencies (C/C++
    only).

  snyk monitor
    Snapshot and continuously monitor a project for open source vulnerabilities and license issues.

  snyk container
    Test container images for vulnerabilities.

  snyk iac
    Commands to find and manage security issues in Infrastructure as Code files.

  snyk code
    Find security issues using static code analysis.

  snyk log4shell
    Find Log4Shell vulnerability.

  snyk config
    Manage Snyk CLI configuration.

  snyk policy
    Display the .snyk policy for a package.

  snyk ignore
    Modify the .snyk policy to ignore stated issues.

Debug
  Use -d option to output the debug logs.

Configure the Snyk CLI
  You can use environment variables to configure the Snyk CLI and also set variables to configure the
  Snyk CLI to connect with the Snyk API. See Configure the Snyk CLI 
  https://docs.snyk.io/features/snyk-cli/configure-the-snyk-cli


参考

[snyk-docs] Snyk CLI のインストールとアップデート (Install or update the Snyk CLI) - Qiita