Troubleshooting Guidelines

This page documents the different troubleshooting resources you can use and provides examples of how to troubleshoot typical problem scenarios.

Troubleshooting Resources

You can use the following resources to troubleshoot issues:

Tools bundled with the SDK:

• apitest

• ZExplore Development Interface (ZEDI)

External tools:

• Evt2Txt—This tool converts the Data ONTAP generated event file to text format.

• Ethereal—This is a network analysis tool.

SDK Help guide:

See the following sections in this Help to collect information you need to troubleshoot:

• Version Matrix for Data ONTAP APIs and DataFabric Manager APIs for the OnCommand Package

• Release History

• Known Issues

• API documentation for Data ONTAP and DataFabric Manager for the OnCommand Package

Log files:

• Audit Logs—Provides Data ONTAP API invocation history, which is at /etc/log/auditlog. For DataFabric Manager, you can find the invocation history at <dfm-server-install-directory>/log/.

• EMS Logs—Provides Data ONTAP API processing information and is at /etc/log/ems.

• Syslog—Provides Data ONTAP API error messages and is at /etc/messages.

• DFM server Logs—Provides error logging information about DataFabric Manager APIs for the OnCommand Package failure. The log is at <dfm-server-install-directory>/log/.

The following list provides some examples of how to troubleshoot typical problem scenarios:

1. While requesting large data, the client application does not receive complete data from the Data ONTAP API or the DataFabric Manager APIs for the OnCommand Package, and the application does not receive any error message.

Possible Causes:

1. The server takes a longer time to process the request because of the large amount of data. This might trigger the server to reset the connection.

2. The reset packets are received at the transport layer of the client host, which is not handled at the API layer of the client application. In this case, the client does not receive the relevant error code.

Diagnosis:

Begin a TCP packet trace using tools such as Ethereal, and check for any error messages from the server.

Workaround:

1. Increase the connection timeout at the server.

2. Use the iterative APIs when requesting a large amount of data (for example,  thousands of qtrees).

2. The client application gets the error message 'Unable to find API'.

Possible Causes:

1. An older version of Data ONTAP is used with a newer version of the SDK that has new Data ONTAP API support. Or, the OnCommand Package version that you are using does not support the API you are trying to invoke. For more information, see the Release History and API documentation.

2. The SDK documentation erroneously includes an unsupported API.

Diagnosis:

1. Check the Data ONTAP version used, and verify whether that API is compatible with the Data ONTAP version. Similarly, check the supported DataFabric Manager APIs for the OnCommand Package. To do this, see the section Guide to Product APIs in the Table of Contents.

2. Try using that API with the apitest tool to check whether there are any SDK documentation errors.

3. The client application gets the error message 'No such file or directory' (error number: 13001).

Possible Causes:

1. There is a problem in the API handler code in Data ONTAP.

2. The application is sending the wrong input (for example, the wrong iteration tag input to the iterative APIs).

3. Race condition in the iterative APIs.

Diagnosis:

1. Check the Audit Log (/etc/log/auditlog) for API invocation history. It hints about possible concurrent calls of APIs leading to race conditions.

2. Check the EMS Log (/etc/log/ems) for any errors that occurred while processing the API requests (for example, the wrong iteration tag).

Workaround:

Upgrade to Data ONTAP 7.3. Race condition of the iterative APIs issue is fixed in Data ONTAP 7.3.

4. The storage system crashes when the client application sends a request to it.

Possible Causes:

1. The API request takes too long to process.

2. An API bug is exposed because of improper input by the application.

Diagnosis:

 For further guidance on troubleshooting, see Support.

Workaround:

1. Use iterative APIs when requesting possibly a huge amount of data (for example, requesting data from thousands of qtrees).

2. Use Data ONTAP 7.3 or Data ONTAP 7.2.3P1.

5. The client application hangs.

Possible Causes:

1. The server reboots during API processing and the application is using blocking connection APIs.

2. Network failure between the client and the server.

Diagnosis:

1. In case of a failed Data ONTAP API call, check uptime of the storage system (using the uptime command). If the uptime indicates that the storage system booted recently, then check whether there was a recent crash in the core file at /etc/crash/core-*. For further guidance on troubleshooting, see Support.

2. In case of a failed call for DataFabric Manager APIs for the OnCommand Package, check if there was a crash recently at the core file in <dfm-server-install-directory>/log/core-*. For further guidance on troubleshooting, see Support.

3. If there is no crash or no API-related message in the crash stack trace, then it might be a power failure or a network-related problem.

Workaround:

1. Restart the client application.

2. Use non-blocking calls in C.

For Data ONTAP API, use the set-time-out() function to change blocking connection API to non-blocking connection API.

For DataFabric Manager APIs for the OnCommand Package, you can use the na_server_set_timeout() function to change blocking connection API to non-blocking connection API.

6. If the Data ONTAP API input schema is not correct,  the API execution fails with the following error messages: Missing input error (error number: 13006), or Extra input error (error number: 13115).

Possible Causes:

The API input schema validator sends this error message if the API input does not match the input parameters set for that API.

The API input schema validator sends the Missing input error (error number: 13006) when there are missing input parameters. It sends the Extra input error (error number: 13115) when the input contains extra parameters.

Diagnosis:

1. Verify that all the input parameters in your application or script match with the API prototype.

2. Update the application or script if there is a mismatch.

Workaround:

Data ONTAP API failure, disable the API input schema validator. To disable, use the following commands:

$ priv set advanced

$ registry set state.api.schema_input_validate.enable off

$ priv set admin

You can use this workaround for Data ONTAP 7.3.1 and later.

7. If the DataFabric Manager APIs for the OnCommand Package input schema is not correct,  the API execution fails with the following error messages: Missing input error (error number: 13006), or Extra input error (error number: 22).

Possible Causes:

The API input schema validator sends this error message if the API input does not match the input parameters set for that API.

The API input schema validator sends the Missing input error (error number:13006) when there are missing input parameters. It sends the Extra input error (error number: 22) when the input contains extra parameters.

Diagnosis:

1. Verify that all the input parameters in your application or script match with the API prototype.

2. Update the application or script if there is a mismatch.

Workaround:

In case of a DataFabric Manager APIs for the OnCommand Package, there is no workaround for this missing input or extra input error.

8. If the API output schema is not correct, the API execution fails with the following error messages: Missing output error (error number: 14513), or Extra output error (error number: 14514).

Possible Causes:

The API output schema validator sends this error message if the API output does not match the input parameters set for that API.

The API output schema sends the Missing output error (error number: 14513) when there are missing output parameters. It sends the Extra output error (error number: 14514) when the output contains extra parameters.

Diagnosis:

This is a bug either in the API implementation or in the API documentation. Contact NetApp technical support and inform them about the bug.

Workaround:

Disable the API output schema validator. You can use the following workaround for Data ONTAP 7.3.1 and Data ONTAP 7.3:

1. For Data ONTAP 7.3.1 and later, use the following commands to disable the API output schema validator:

$ priv set advanced

$ registry set state.api.schema_output_validate.enable off

$ priv set admin

 

2. For Data ONTAP 7.3, use the following commands to disable the API output schema validator:

$ priv set advanced

$ registry set state.api.schema_validate.enable off

$ priv set admin

9. When you use -s option with the apitest utility, it fails with an error message.

Possible Cause:

Net::SSLeay is not available to the apitest utility.

Workaround:

Install Net::SSLeay, or update the PATH so that Net::SSLeay is accessible from apitest.

10. If an application takes more than ten minutes to retrieve the complete iterator API output, the API query fails with an error message "Removing iterator file that is too old".

Possible Causes:

The Data ONTAP API iterator files are deleted from Data ONTAP server after a specified time, which is 600 seconds by default. An application must access the API iterator files before the iterator files are deleted.

Diagnosis:

The following errors message are reported at the EMS Log (/etc/log/ems): errorDescription="Removing iterator file that is too old".

Workaround:

You can avoid this error by completing the iterative API queries before the iterator file is deleted at the server.

If the application cannot process these queries within the specified time, you must modify the application to divide the API query process into two steps. In the first step, the application should complete the queries and save the output locally.  In the second step, the client should process the saved output.

Related Topics

• apitest