[FIXED] Trying to understand how describe_instances() work

Issue

Trying to understand how describe_instances() work Here is a code which gives me instances ids of all the instances I have currently 3 so 3 instance id I get

    import boto3 
    from pprint import pprint

    ec2=boto3.client('ec2') 
    response=ec2.describe_instances()
    instancelist = [] 
   for reservation in (response["Reservations"]):
        for instance in reservation["Instances"]:
            instancelist.append(instance["InstanceId"]) print (instancelist)

gives me output

['i-03e7f6391a0f523ee', 'i-0e12c8dad5fb6d8c5', 'i-002adcd0913e4d673']

But if I write following

import boto3
from pprint import pprint
ec2=boto3.client('ec2')
response=ec2.describe_instances()
for x in response:
    print (x)
    print ("in for loop")

I just get following output

Reservations
in for loop
ResponseMetadata
in for loop

What I was expecting was instance id’s of instance. Now I wrote following

import boto3
from pprint import pprint

ec2=boto3.client('ec2')
response=ec2.describe_instances()

print (response["Reservations"][0]["Instances"][0]["InstanceId"])
print (response["Reservations"][1]["Instances"][0]["InstanceId"])
print (response["Reservations"][1]["Instances"][1]["InstanceId"])

I get output

i-03e7f6391a0f523ee i-0e12c8dad5fb6d8c5 i-002adcd0913e4d673

the above print statement does not work if written as below

print (response["Reservations"][2]["Instances"][0]["InstanceId"])

It gives error

    print (response["Reservations"][2]["Instances"][1]["InstanceId"])
IndexError: list index out of range

What I am not able to understand is how are the values being returned by describe_instances method in the statement

response=ec2.describe_instances()

Why the above for loop where I used x does not give output. I read the documentation here https://boto3.readthedocs.io/en/latest/reference/services/ec2.html#EC2.Client.describe_instances and as far as I understand ec2.describe_instances() is returning a dictionary() Contains the output of DescribeInstances. • Reservations (list) -- Zero or more reservations. So with that understanding I thought I should be able to iterate over list members via

response["Reservations"][0]
response["Reservations"][1]
response["Reservations"][2]

But this is giving me error

print(response["Reservations"][2])
IndexError: list index out of range

I am unable to understand the response which is coming via ec2.describe_instances()

Output of print (response) as asked in below answer

    {'Reservations': [

    {'Groups': [], 'Instances': [{'AmiLaunchIndex': 0, 'ImageId': 'ami-d783a9b8', 

    'InstanceId': 'i-03e7f6391a0f523ee', 'InstanceType': 't2.micro', 'KeyName': 'datastructutre key', 

    'LaunchTime': datetime.datetime(2018, 8, 25, 10, 3, 53, tzinfo=tzutc()), 'Monitoring': {'State': 

    'disabled'}, 'Placement': {'AvailabilityZone': 'ap-south-1a', 'GroupName': '', 'Tenancy': 

    'default'}, 'PrivateDnsName': 'ip-172-31-20-16.ap-south-1.compute.internal', 'PrivateIpAddress': 

    '172.31.20.16', 'ProductCodes': [], 'PublicDnsName': '', 'State': {'Code': 80, 'Name': 

    'stopped'}, 'StateTransitionReason': 'User initiated (2018-08-25 10:07:17 GMT)', 'SubnetId': 

    'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60', 'Architecture': 'x86_64', 'BlockDeviceMappings': 

    [{'DeviceName': '/dev/xvda', 'Ebs': {'AttachTime': datetime.datetime(2018, 8, 1, 4, 28, 52, 

    tzinfo=tzutc()), 'DeleteOnTermination': True, 'Status': 'attached', 'VolumeId': 'vol-

    09716d3308f44c63f'}}], 'ClientToken': '', 'EbsOptimized': False, 'EnaSupport': True, 

    'Hypervisor': 'xen', 'NetworkInterfaces': [{'Attachment': {'AttachTime': datetime.datetime(2018, 

    8, 1, 4, 28, 52, tzinfo=tzutc()), 'AttachmentId': 'eni-attach-08d060230b617ca70', 

    'DeleteOnTermination': True, 'DeviceIndex': 0, 'Status': 'attached'}, 'Description': '', 

    'Groups': [{'GroupName': 'launch-wizard-1', 'GroupId': 'sg-0e81c2a33e1039f58'}, {'GroupName': 

    'default', 'GroupId': 'sg-40e5492a'}], 'Ipv6Addresses': [], 'MacAddress': '02:5a:17:52:69:a6', 

    'NetworkInterfaceId': 'eni-0146aab6d9503bf47', 'OwnerId': '11000101010', 'PrivateDnsName': 'ip-

    172-31-20-16.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.20.16', 

    'PrivateIpAddresses': [{'Primary': True, 'PrivateDnsName': 'ip-172-31-20-16.ap-south-

    1.compute.internal', 'PrivateIpAddress': '172.31.20.16'}], 'SourceDestCheck': True, 'Status': 

    'in-use', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60'}], 'RootDeviceName': 

    '/dev/xvda', 'RootDeviceType': 'ebs', 'SecurityGroups': [{'GroupName': 'launch-wizard-1', 

    'GroupId': 'sg-0e81c2a33e1039f58'}, {'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 

    'SourceDestCheck': True, 'StateReason': {'Code': 'Client.UserInitiatedShutdown', 'Message': 

    'Client.UserInitiatedShutdown: User initiated shutdown'}, 'VirtualizationType': 'hvm', 

    'CpuOptions': {'CoreCount': 1, 'ThreadsPerCore': 1}}], 'OwnerId': '11000101010', 'ReservationId': 

    'r-0571937a9ea83fac4'}, 

  {'Groups': [], 'Instances': [

{'AmiLaunchIndex': 0, 'ImageId': 'ami-00b6a8a2bd28daf19', 'InstanceId': 'i-0e12c8dad5fb6d8c5', 'InstanceType': 't2.micro', 'KeyName': 'datastructutre key', 'LaunchTime': datetime.datetime(2018, 8, 25, 10, 3, 53, tzinfo=tzutc()), 'Monitoring': {'State': 'disabled'}, 'Placement': {'AvailabilityZone': 'ap-south-1a', 'GroupName': '', 'Tenancy': 'default'}, 'PrivateDnsName': 'ip-172-31-28-39.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.28.39', 'ProductCodes': [], 'PublicDnsName': '', 'State': {'Code': 80, 'Name': 'stopped'}, 'StateTransitionReason': 'User initiated (2018-08-25 10:07:17 GMT)', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60', 'Architecture': 'x86_64', 'BlockDeviceMappings': [{'DeviceName': '/dev/xvda', 'Ebs': {'AttachTime': datetime.datetime(2018, 8, 22, 21, 33, 44, tzinfo=tzutc()), 'DeleteOnTermination': True, 'Status': 'attached', 'VolumeId': 'vol-01d5ce67c9f1b081e'}}], 'ClientToken': '', 'EbsOptimized': False, 'EnaSupport': True, 'Hypervisor': 'xen', 'NetworkInterfaces': [{'Attachment': {'AttachTime': datetime.datetime(2018, 8, 22, 21, 33, 44, tzinfo=tzutc()), 'AttachmentId': 'eni-attach-0cbdea888315049ae',  'DeleteOnTermination': True, 'DeviceIndex': 0, 'Status': 'attached'}, 'Description': '',  'Groups': [{'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 'Ipv6Addresses': [], 'MacAddress': '02:f4:b8:bd:84:26', 'NetworkInterfaceId': 'eni-0d99a7669a1e4b9db', 'OwnerId': '11000101010', 'PrivateDnsName': 'ip-172-31-28-39.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.28.39', 'PrivateIpAddresses': [{'Primary': True, 'PrivateDnsName': 'ip-172-31-28-39.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.28.39'}], 'SourceDestCheck': True, 'Status': 'in-use', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60'}], 'RootDeviceName': '/dev/xvda', 'RootDeviceType': 'ebs', 'SecurityGroups': [{'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 'SourceDestCheck': True, 'StateReason': {'Code': 'Client.UserInitiatedShutdown', 'Message': 'Client.UserInitiatedShutdown: User initiated shutdown'}, 'VirtualizationType': 'hvm', 'CpuOptions': {'CoreCount': 1, 'ThreadsPerCore': 1}}, 

{'AmiLaunchIndex': 1, 'ImageId': 'ami-00b6a8a2bd28daf19', 'InstanceId': 'i-002adcd0913e4d673', 'InstanceType': 't2.micro', 'KeyName': 'datastructutre key', 'LaunchTime': datetime.datetime (2018, 8, 25, 10, 3, 53, tzinfo=tzutc()), 'Monitoring': {'State': 'disabled'}, 'Placement': {'AvailabilityZone': 'ap-south-1a', 'GroupName': '', 'Tenancy': 'default'}, 'PrivateDnsName': 'ip-172-31-30-108.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.30.108', 'ProductCodes': [], 'PublicDnsName': '', 'State': {'Code': 80, 'Name': 'stopped'}, 'StateTransitionReason': 'User initiated (2018-08-25 10:07:17 GMT)', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60', 'Architecture': 'x86_64', 'BlockDeviceMappings': [{'DeviceName': '/dev/xvda', 'Ebs': {'AttachTime': datetime.datetime(2018, 8, 22, 21, 33, 44, tzinfo=tzutc()), 'DeleteOnTermination': True, 'Status': 'attached', 'VolumeId': 'vol-0f0c49cc912a083f3'}}], 'ClientToken': '', 'EbsOptimized': False, 'EnaSupport': True, 'Hypervisor': 'xen', 'NetworkInterfaces': [{'Attachment': {'AttachTime': datetime.datetime(2018, 8, 22, 21, 33, 44, tzinfo=tzutc()), 'AttachmentId': 'eni-attach-0f35d1842b76cff9a', 'DeleteOnTermination': True, 'DeviceIndex': 0, 'Status': 'attached'}, 'Description': '', 'Groups': [{'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 'Ipv6Addresses': [], 'MacAddress': '02:21:d4:20:22:c6', 'NetworkInterfaceId': 'eni-0e3797492dc4e3299', 'OwnerId': '11000101010', 'PrivateDnsName': 'ip-172-31-30-108.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.30.108', 'PrivateIpAddresses': [{'Primary': True, 'PrivateDnsName': 'ip-172-31-30-108.ap-south-1.compute.internal', 'PrivateIpAddress': '172.31.30.108'}], 'SourceDestCheck': True, 'Status': 'in-use', 'SubnetId': 'subnet-d3fdbabb', 'VpcId': 'vpc-08356c60'}], 'RootDeviceName': '/dev/xvda', 'RootDeviceType': 'ebs', 'SecurityGroups': [{'GroupName': 'default', 'GroupId': 'sg-40e5492a'}], 'SourceDestCheck': True, 'StateReason': {'Code': 'Client.UserInitiatedShutdown', 'Message': 'Client.UserInitiatedShutdown: User initiated shutdown'}, 'VirtualizationType': 'hvm', 'CpuOptions': {'CoreCount': 1, 'ThreadsPerCore': 1}}], 'OwnerId': 'ReservationId': 'r-05cff59b2524ed79c'}], 

'ResponseMetadata': {'RequestId': 'fc80ae94-dd46-4c71-93f5-a38d6ede800c', 'HTTPStatusCode': 200, 'HTTPHeaders': {'content-type':'text/xml;charset=UTF-8', 'transfer-encoding': 'chunked', 'vary': 'Accept-Encoding', 'date': 'Sat, 25 Aug 2018 17:13:51 GMT', 'server': 'AmazonEC2'}, 'RetryAttempts': 0}}

You can note {'Groups': [], 'Instances': [{'AmiLaunchIndex': 0, line 2 times in above output and line 'AmiLaunchIndex': 1, 'ImageId': 'ami-00b6a8a2bd28daf19' two times while for the first instance it is 'AmiLaunchIndex': 0, 'ImageId': 'ami-d783a9b8', I had created these instances via run_instance method() as you can see in the print (response) output has come only but this is not giving a clear picture or I am unable to understand where are three instances in this this is what is not clear to me.

Solution

Basically, the response object will return one or many Reservation objects.

response=ec2.describe_instances()

You don't really have to worry about Reservations, other than for the purpose of iterating them. To delve into more detail, you could have a look at this past question,

https://serverfault.com/questions/749118/aws-ec2-what-is-a-reservation-id-exactly-and-what-does-it-represent

So your first block of code is an appropriate way of listing the instance ID's of all EC2 instances.

The second block of code is iterating through the keys in the response object, which is a dictionary,

response=ec2.describe_instances()
for x in response:
    print(x)

Here's the Boto documentation for the EC2 service, where you can find more info on the response object,

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_instances

Answered By - AYA