Troubleshooting

This page contains various tips which can help you troubleshoot and debugcode with interfaces with libcloud.

Debugging

Note

If you are sharing debug output on any public medium such as our IRCchannel or an issue tracker using Pastebin, Github Gists or a similarservice, make sure to remove your credentials and any other data youconsider private from the output.

Libcloud has a special debug mode which when enabled, logs all the outgoingHTTP requests and all the incoming HTTP responses. Output also includes cURLcommands which can be used to re-produce the requests.

When this mode is enabled and paramiko library is installed (used fordeployment), paramiko library log level is set to DEBUG which helps withdebugging the deployment related issues.

To make the debugging easier, Libcloud will also automatically decompress theresponse body (if compressed) before logging it.

To enable it, set LIBCLOUD_DEBUG environment variable and make it pointto a file where the debug output should be saved.

If the API returns JSON or XML in the response body which is not humanfriendly, you can also set LIBCLOUD_DEBUG_PRETTY_PRINT_RESPONSEenvironment variable which will cause the JSON or XML to be beautified/ formated so it’s easier for humans to read it. Keep in mind that thisonly works for non-chunked responses.

Example 1 - Logging output to standard error

Note

Some Linux distributions don’t allow /dev/{stderr,stdout} to be opened inappend mode which means paramiko logger will return and error that itcan’t log to that file.In such scenario, it’s recommended you specify actual file path and thenyou can use “tail -F” in another terminal window to view the output inreal-time.

If you want the output to be logged to the standard error (onLinux) you can set it to /dev/stderr:

  1. LIBCLOUD_DEBUG=/dev/stderr python my_script.py

Example output:

  1. # -------- begin 4431824872 request ----------
  2. curl -i -X GET -H 'Host: s3.amazonaws.com' -H 'X-LC-Request-ID: 4431824872' -H 'Content-Length: 0' -H 'User-Agent: libcloud/0.6.0-beta1 (Amazon S3 (standard))' 'https://s3.amazonaws.com:443/?AWSAccessKeyId=foo&Signature=bar'
  3. # -------- begin 4431824872:4431825232 response ----------
  4. HTTP/1.1 200 OK
  5. X-Amz-Id-2: 1234
  6. Server: AmazonS3
  7. Transfer-Encoding: chunked
  8. X-Amz-Request-Id: FFFFFFFFFF
  9. Date: Tue, 01 Nov 2011 22:29:11 GMT
  10. Content-Type: application/xml
  11.  
  12. 171
  13. <?xml version="1.0" encoding="UTF-8"?>
  14. <ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>sada8932dsa8d30i</ID><DisplayName>kami</DisplayName></Owner><Buckets><Bucket><Name>test34324323</Name><CreationDate>2011-11-01T22:17:23.000Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>
  15. 0
  16.  
  17. # -------- end 4431824872:4431825232 response ----------

Example 2 - Making JSON / XML response human friendly

Non-formatted JSON response:

  1. LIBCLOUD_DEBUG=/dev/stderr python my_script.py
  1. # -------- begin 23125648:23160304 response ----------
  2. HTTP/1.1 200 OK
  3. Content-Length: 1572
  4. X-Compute-Request-Id: req-79ab42d8-a959-44eb-8dec-bc9458b2f4b3
  5. Server: nginx/1.4.7
  6. Connection: keep-alive
  7. Date: Sat, 06 Sep 2014 14:13:37 GMT
  8. Content-Type: application/json
  9.  
  10. {"servers": [{"status": "ACTIVE", "updated": "2014-09-06T14:13:32Z", "hostId": "561d56de25c177c422278d7ca5f8b210118348040b12afbad06f278a", "addresses": {"internet-routable": [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:3f:c0:a1", "version": 4, "addr": "10.100.100.101", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": "http://nova/v2/d3b31ebfd32744d19d848f3e9c351869/servers/deb35f96-be41-431e-b931-6e615ec720f4", "rel": "self"}, {"href": "http://nova/d3b31ebfd32744d19d848f3e9c351869/servers/deb35f96-be41-431e-b931-6e615ec720f4", "rel": "bookmark"}], "key_name": null, "image": {"id": "e9537ddd-6579-4473-9898-d211ab90f6d3", "links": [{"href": "http://nova/d3b31ebfd32744d19d848f3e9c351869/images/e9537ddd-6579-4473-9898-d211ab90f6d3", "rel": "bookmark"}]}, "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "active", "OS-SRV-USG:launched_at": "2014-09-06T14:13:32.000000", "flavor": {"id": "90c2a137-611b-4dd2-9d65-d4a0b0858531", "links": [{"href": "http://nova/d3b31ebfd32744d19d848f3e9c351869/flavors/90c2a137-611b-4dd2-9d65-d4a0b0858531", "rel": "bookmark"}]}, "id": "deb35f96-be41-431e-b931-6e615ec720f4", "security_groups": [{"name": "default"}], "OS-SRV-USG:terminated_at": null, "OS-EXT-AZ:availability_zone": "nova", "user_id": "06dda7c06aa246c88d7775d02bc119ac", "name": "test lc 2", "created": "2014-09-06T14:13:12Z", "tenant_id": "d3b31ebfd32744d19d848f3e9c351869", "OS-DCF:diskConfig": "MANUAL", "os-extended-volumes:volumes_attached": [], "accessIPv4": "", "accessIPv6": "", "progress": 0, "OS-EXT-STS:power_state": 1, "config_drive": "", "metadata": {}}]}
  11. # -------- end 23125648:23160304 response ----------

Human friendly formatted JSON response:

  1. LIBCLOUD_DEBUG=/dev/stderr LIBCLOUD_DEBUG_PRETTY_PRINT_RESPONSE=1 python my_script.py
  1. # -------- begin 41102928:41133624 response ----------
  2. HTTP/1.1 200 OK
  3. Content-Length: 1572
  4. X-Compute-Request-Id: req-3ce8b047-55cd-4e20-bfeb-b65619696aec
  5. Server: nginx/1.4.7
  6. Connection: keep-alive
  7. Date: Sat, 06 Sep 2014 14:14:38 GMT
  8. Content-Type: application/json
  9.  
  10. {
  11.     "servers": [
  12.         {
  13.             "OS-DCF:diskConfig": "MANUAL",
  14.             "OS-EXT-AZ:availability_zone": "nova",
  15.             "OS-EXT-STS:power_state": 1,
  16.             "OS-EXT-STS:task_state": null,
  17.             "OS-EXT-STS:vm_state": "active",
  18.             "OS-SRV-USG:launched_at": "2014-09-06T14:13:32.000000",
  19.             "OS-SRV-USG:terminated_at": null,
  20.             "accessIPv4": "",
  21.             "accessIPv6": "",
  22.             "addresses": {
  23.                 "internet-routable": [
  24.                     {
  25.                         "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:3f:c0:a1",
  26.                         "OS-EXT-IPS:type": "fixed",
  27.                         "addr": "10.100.100.101",
  28.                         "version": 4
  29.                     }
  30.                 ]
  31.             },
  32.             "config_drive": "",
  33.             "created": "2014-09-06T14:13:12Z",
  34.             "flavor": {
  35.                 "id": "90c2a137-611b-4dd2-9d65-d4a0b0858531",
  36.                 "links": [
  37.                     {
  38.                         "href": "http://nova/d3b31ebfd32744d19d848f3e9c351869/flavors/90c2a137-611b-4dd2-9d65-d4a0b0858531",
  39.                         "rel": "bookmark"
  40.                     }
  41.                 ]
  42.             },
  43.             "hostId": "561d56de25c177c422278d7ca5f8b210118348040b12afbad06f278a",
  44.             "id": "deb35f96-be41-431e-b931-6e615ec720f4",
  45.             "image": {
  46.                 "id": "e9537ddd-6579-4473-9898-d211ab90f6d3",
  47.                 "links": [
  48.                     {
  49.                         "href": "http://nova/d3b31ebfd32744d19d848f3e9c351869/images/e9537ddd-6579-4473-9898-d211ab90f6d3",
  50.                         "rel": "bookmark"
  51.                     }
  52.                 ]
  53.             },
  54.             "key_name": null,
  55.             "links": [
  56.                 {
  57.                     "href": "http://nova/v2/d3b31ebfd32744d19d848f3e9c351869/servers/deb35f96-be41-431e-b931-6e615ec720f4",
  58.                     "rel": "self"
  59.                 },
  60.                 {
  61.                     "href": "http://nova/d3b31ebfd32744d19d848f3e9c351869/servers/deb35f96-be41-431e-b931-6e615ec720f4",
  62.                     "rel": "bookmark"
  63.                 }
  64.             ],
  65.             "metadata": {},
  66.             "name": "test lc 2",
  67.             "os-extended-volumes:volumes_attached": [],
  68.             "progress": 0,
  69.             "security_groups": [
  70.                 {
  71.                     "name": "default"
  72.                 }
  73.             ],
  74.             "status": "ACTIVE",
  75.             "tenant_id": "d3b31ebfd32744d19d848f3e9c351869",
  76.             "updated": "2014-09-06T14:13:32Z",
  77.             "user_id": "06dda7c06aa246c88d7775d02bc119ac"
  78.         }
  79.     ]
  80. }
  81. # -------- end 41102928:41133624 response ----------

Non-formatted XML response:

  1. LIBCLOUD_DEBUG=/dev/stderr python my_script.py
  1. # -------- begin 33145616:33126160 response ----------
  2. HTTP/1.1 200 OK
  3. X-Amzn-Requestid: e84f62d0-368e-11e4-820b-8bf013dc269e
  4. Date: Sun, 07 Sep 2014 13:00:13 GMT
  5. Content-Length: 457
  6. Content-Type: text/xml
  7.  
  8. <?xml version="1.0"?>
  9. <ListHostedZonesResponse xmlns="https://route53.amazonaws.com/doc/2012-02-29/"><HostedZones><HostedZone><Id>/hostedzone/Z14L0C73CHH1DN</Id><Name>example1.com.</Name><CallerReference>41747982-568E-0DFC-8C11-71C23757C740</CallerReference><Config><Comment>test</Comment></Config><ResourceRecordSetCount>9</ResourceRecordSetCount></HostedZone></HostedZones><IsTruncated>false</IsTruncated><MaxItems>100</MaxItems></ListHostedZonesResponse>
  10. # -------- end 33145616:33126160 response ----------

Human friendly formatted XML response:

  1. LIBCLOUD_DEBUG=/dev/stderr LIBCLOUD_DEBUG_PRETTY_PRINT_RESPONSE=1 python my_script.py
  1. # -------- begin 19444496:19425040 response ----------
  2. HTTP/1.1 200 OK
  3. X-Amzn-Requestid: 01c02441-368f-11e4-b616-9b9bd7509a8f
  4. Date: Sun, 07 Sep 2014 13:00:56 GMT
  5. Content-Length: 457
  6. Content-Type: text/xml
  7.  
  8. <?xml version="1.0" ?>
  9. <ListHostedZonesResponse xmlns="https://route53.amazonaws.com/doc/2012-02-29/">
  10.     <HostedZones>
  11.         <HostedZone>
  12.             <Id>/hostedzone/Z14L0C73CHH1DN</Id>
  13.             <Name>example1.com.</Name>
  14.             <CallerReference>41747982-568E-0DFC-8C11-71C23757C740</CallerReference>
  15.             <Config>
  16.                 <Comment>test</Comment>
  17.             </Config>
  18.             <ResourceRecordSetCount>9</ResourceRecordSetCount>
  19.         </HostedZone>
  20.     </HostedZones>
  21.     <IsTruncated>false</IsTruncated>
  22.     <MaxItems>100</MaxItems>
  23. </ListHostedZonesResponse>
  24.  
  25. # -------- end 19444496:19425040 response ----------