The draft guideline manuscript was reviewed by a diverse group of experts representing people with HHT, patient advocacy groups, a variety of relevant specialties and guidelines experts. Reviewers were from eight countries and excluded individuals from the centers of current HHT guidelines authors. Reviewers were recommended by the Guidelines Working Group. Those with relevant expertise were invited by the Guidelines chair (MEF). Of the 33 invited reviewers, 21 indicated a willingness to provide a review, 8 declined and 4 did not respond. In total, 20 reviews were completed.

Reviewers were asked to read the confidential draft manuscript and then to answer a survey.  The purpose of the external review was to obtain input on the Guidelines manuscript and the clinical relevance of the recommendations, and to identify potential barriers and enablers to implementation.   Given that the clinical recommendation statements were generated and already finalized at the consensus conference, reviewers were not asked for revisions of the recommendation statements.

The survey instrument was hosted on Survey Monkey. Two reviewers provided feedback in separate emails in addition to completing the survey instrument.

Responses to overall questions (yes/no closed questions):

Question	Total responses N (%)	Yes  N (%)	No N (%)	Not applicable N (%)
Would you make use of this guideline in your professional decisions?	20 (100%)	15 (75%)	2 (10%)	3 (15%)
Are these guidelines flexible enough to allow for clinical judgement?	20 (100%)	17 (85)	2 (10%)	1 (5%)
I would recommend these guidelines for use in practice.	19 (95%)	18 (95%)	1 (5%)	0 (0%)

 

One reviewer, with specific expertise in the GRADE methodology, highlighted areas where transparency of the recommendation decision process could be increased.

 

Ratings of guideline quality: 

Reviewers were asked to rate the quality of the Guidelines using a 1 to 5 scale (1=poor quality,  5 =high quality). High quality was defined as impactful, rigorous methodology, clear and clinically relevant.  Poor quality was defined a low impact, unclear, lack of clinical relevance and weak methodology.

Question	1 (poor)	2	3	4	5 (high)	Weighted average	Total responses
Rate the quality of these guidelines.	0	0	2	5	12	4.63	19

Note: high quality was defined as impactful, rigorous methodology, clear, clinically relevant.

 

Implementation Input:

Reviewers were asked several open-ended questions related to perceived implementation barriers and enablers that the Working Group will consider in planning for dissemination and implementation. Responses included comments and suggestions about systemic and educational barriers and evaluating the impact of the recommendations on daily practice.   When asked which guidelines were likely to have the biggest impact, responses were generally aligned with reviewer specialties. The two reviewers representing patient advocacy organizations perceived Pediatrics, Epistaxis and Pregnancy and Delivery as the topic areas likely to have the biggest impact.

Reviewers’ specific comments on the draft manuscript and guideline panel response

Comments	Responses
Multiple (N = 20) text suggestions that strengthen the background and clinical considerations sections:	Manuscript has been clarified with respect to these comments.
Several topics were raised as relevant but not addressed in these Guidelines (N=8)	Added to future research priorities and future guidelines priorities clinical question.