Background

Accurate financial disclosures by research investigators are required for proper compliance with Federal regulations and evaluation of conflicts of interest. Institutions can analyze publicly available datasets through the Centers for Medicare & Medicaid Services (CMS) Open Payments program to identify gaps in the financial disclosures received from investigators.

Program Description

Data from CMS Open Payments was downloaded and compared to the disclosure data available from the research institution. This was achieved by creating a report in Tableau to assess if the individual was active at the institution and then comparing their disclosed data. This report included a gap calculator to assess the amount of difference between the two data sets. Under-reporting (reporting less remuneration to the institution than was reported in CMS data) and over-reporting (reporting more remuneration to the institution than was reported in CMS data) were displayed. A manual check of any under-reporting was performed by the Conflict of Interest Office and investigators were contacted for their comments.

Program Assessment

It was discovered that the CMS data included some payments that were considered unreportable by institutional standards, and over-reporting was more prevalent at the institution. A total of 117 individuals were identified with potential gaps. Of these, 54% had over-reported in their disclosure to the institution. Initial feedback from investigators indicated a concern that the information reported in the CMS data would contain inaccurate data (though CMS offers a correction opportunity prior to publishing). Several investigators had grant information added to their CMS information, though this funding flowed through the institution. Several investigators were contacted about gaps in their reporting and disclosures were updated with the corrected information. Institutional Policy was used to determined a significance level at a threshold of $5,000.

Limitations

CMS data is reported with an individual’s name, institutional address, and unique Open Payments ID. Though our comparison report used individuals’ names for identification, it is possible that some individuals were missed, due to spelling differences, or inaccuracy of institutional address reported. CMS data only reports on a subset of individuals who potentially participate in research (those with National Provider Identifiers, e.g. physicians, physician assistants, etc.). New CMS datasets are only published annually.

Over-reporting individuals were not actively contacted in this comparison due to the time and labor required to create an inquiry.  

Discussion

The data comparison allowed us to identify disclosure gaps and gain more insight on the publicly available information reported by Open Payments. A method for obtaining the Open Payments ID to better identify investigators is needed. Though the review of Open Payments data is a highly discussed topic within the COI Community, few institutions have engaged in the level of review that our office has undertaken. This comparison was unique to our institution in the level of review and follow up. We have not seen any publications detailing any institution using a similar system, and only one other institution in a national COI network (Association of American Medical College’s Forum on Conflicts of Interest) has presented a comparison. That comparison did not include the investigator outreach.

There have been several articles published by various journalists using this publicly available data to investigate institutions, and the method described here may be useful to institutions who are responsible for reviewing financial disclosures and potential conflicts in health sciences. Though most of the entries in this analysis were over-reporters, we believe the risk mitigation is worth the effort. Institutions employing individuals whose data is presented on Open Payments are required by the federal government to verify the accuracy of this data. Additionally, public perception of an error in the data could negatively affect institution reputation and standing within the community.

After the evaluation of the data, investigators were contacted to clarify discrepancies and provided with educational materials to better disclose their payment. This was in the form of noting payments made through a consulting group with the entity names.

The process of creating this comparison tool took several tries to get to the point of effective use. The first pilot of this comparison was created using an Excel spreadsheet which did not allow for the same level of detail. Utilizing Tableau instead has allowed for a more detailed comparison. The system has allowed us to see the specific details that we want in our report, but also allows us to see the original data in separate window on the same workstation. Designing the initial Tableau template took 4 months, but now further data can imported and analyzed easily.