RTMP does not support HEVC encoded streams. Adobe RTMP specification version 1.0 no details about HEVC, but it is possible to define a video codec ID to use for HEVC. Also it need to add video codec header to libavformat (path of ffmpeg). you can see the example here