The communication failure indicates that your RFC server required for the DQM for SAP product are either not installed or are not running successfully. The flrfcsrv.xml file is installed in the \conf directory of your RFC server installation location. Typically you don't need to edit this file unless instructed to after installation. The necessary jobs are installed as part of the DQM for SAP Data Services Component installation for the Service Pack you are installing, and install into a working Data Services environment/repository.
Make sure the RFC server and the Data Services component installation are for the same Service Pack as the BAdI's installed on the SAP system. the RFC and DS component are full installs and you can install only the one version.
If the RFC is installed, and the RFC has been configured in SM 59, locate the flrfcsrv_log.txt file in the RFC installation location \ logs \ <SID>_trans and <SID>_batch to understand what error the RFC is experiencing when trying to process data.
Its also best practice to run all of the jobs that are installed and configured as real-time services in Data Services Designer prior to starting them as a service, to ensure that they will run. if the jobs do not run in Data Services designer, they will not run steadily as a service, and the RFC server will go into a "hibernating" state.
Please refer to the DQM for SAP Users guide for the Service pack that you are installing for additional information.
