Modifying RNN CuDNN example code to use CUDNN_DATA_INT8 - recurrent-neural-network

Question

The RNN example (RNN_example.cu) that is in cudnn_samples_v7 is set up to use CUDNN_DATA_FLOAT. I'd like to modify it to use CUDNN_DATA_INT8. When I globally made this change, compiled and run, I get the following runtime errors:
$ ./RNN_int8 20 2 512 64 0
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 285
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 302
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 309
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 310
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 326
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 328
CUDA Error: out of memory RNN_example_int8.cu 330
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 482
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 516
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 541
For example, the error at line 309 is running this code:
cudnnErrCheck(cudnnSetFilterNdDescriptor(wDesc, CUDNN_DATA_INT8, CUDNN_TENSOR_NCHW, 3, dimW));
I suspect the format type (CUDNN_TENSOR_HCHW) isn't correct for the data type - is that the correct hypothesis? If so, what format is needed here?

Answer 1

CuDNN currently doesn't support INT8 RNN, we suggest you use FP16 input output to achieve better performance than float. If you use Volta be sure to turn on CUDNN_TENSOR_OP_MATH to get hardware acceleration.
All supported config combinations of cuDNN RNN can be found here
https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#features-of-rnn-functions
Let us know if you have any other issues!