GRADEpro GDT - an introduction to the system

GRADEpro GDT

GRADEpro GDT, formerly named GDT (Guideline Development Tool) or G2DT (GRADE Guideline Development Tool) is an application designed for the authors of health care guidelines. It is also a complete re-design and re-write of the former, Windows-only tool GRADEpro v. 3.0, extended with a range of cooperative and authoring functionalities, designed in collaboration with a broad forum of methodologists from the GRADE Working Group, DECIDE project and more.

In this document, the names “GRADEpro GDT” and “GRADEpro” are – unless specified otherwise – used as synonyms.

The basics

GRADEpro GDT aims to support the complete process of developing health care guidelines, starting from the team composition and management and subject inception to the generation of the recommendations, authoring of the final recommendation document and the dissemination of the guidelines.

While the intention is to enable the users to carry on the complete process of guideline development under one roof of the single application, the system remains open, allowing usage of third-party tools, plugins and services. Currently, the systematic review part of the guideline development (from search to meta-analysis) is done entirely using third-party tools, including integration with RevMan.

GRADEpro is a complex web application, publically available at http://guidelinedevelopment.org and http://gradepro.org , allowing free and unrestricted use for any non-commercial projects. The commercial use requires a license (all the income from the commercial licenses is re-directed to further development of the tool and maintaining its free availability).

GRADEpro available at the links above is a shared-database system, which – while maintaining quite high security standards – cannot guarantee that the data will never be accessed by the unprivileged users (due to e.g. bugs in the used technologies, hacker activities or users themselves sharing the data with the other users).

In cases, where privacy of the data is of utmost concern, the operator of GRADEpro (Evidence Prime) provides upon requests two forms of private installations that maintain separation of the data from the other users, as well as encryption of the communication channels. These installations also allow commercial use.

The main functions of GRADEpro

GRADEpro aims at covering the complete guideline development process, including support for the GRADE Working Group and Cochrane Collaboration standards:

• Team management
• Scope management including collaborative question and outcome generation
• Conflict of Interest management
• Evidence table creation including Interactive Summary of Findings
• Evidence-to-Decision process facilitation including Interactive EtD
• Final document authoring and generation
• Dissemination of guidelines by information exchange and end-user document and mobile application generation
• Exchange of data with a variety of external systems

GRADEpro has a highly modular and extensible structure and can be easily adapted for particular customer needs. For instance, the Ministry of Health of Saudi Arabia requested a dedicated version featuring not only their branding, but also customized reports and collaboration forms.

GRADEpro has been designed to allow changes and sustainability over time. All data will survive changes in the processes of developing systematic reviews and health care guidelines, as well as changes in data structures and the data standards, by smart automated and semi-manual mapping.

GRADEpro has been designed to enable exchange of data with third party tools and to work seamlessly with external systems and common evidence databases, including the Database of Evidence Profiles that is being developed by Evidence Prime for the European Commission sponsored DECIDE research project.

Technical aims

GRADEpro was designed as a professional tool for collaborative as well as individual work of experts, and has a number of technical and functional attributes rarely found in current health care systems. These include, but are not limited to: 

• Support for any operating system, desktop computer and mobile device,  
• Ability to work in online and offline modes with automated synchronization,  
• Collaborative real-time editing,  
• Exchange of data with various external systems (e.g. Cochrane Collaboration’s Review Manager [RevMan], Database of Evidence Profiles, EMR systems etc.), 
• Flexibility and very high adaptability to the needs of organizations and individuals, 
• Support for multiple languages
• High stability and security.

All this is possible thanks to a very well designed software architecture utilizing the full power of the modern browsers and HTML5 technologies, bringing the users the full power of a complex application in a browser or on mobile devices.

Integrated GRADE handbook

GRADEpro also provides an instant access to the GRADE Handbook, a comprehensive guide to the best practices in the guideline development process (the GRADE approach).

Screenshots

The screenshots illustrate some of the features of GRADEpro that can be accessed at http://www.gradepro.org.

GDT Primer

This chapter provides a brief introduction to basic functionalities of the GRADEpro. For even more comprehensive guidance please refer to the tutorials, documentation and help in the tool itself (http://www.gradepro.org).

The site and login

GRADEpro is a web-based software, designed to help the users develop guidelines in an organized way and according to the best practices available.

The system is available at http://www.gradepro.org

After clicking “log in” for the first time, the system loads and requests the user to register an account:

The Project Management screen

One of the first screens contains a list of the user projects (own or shared by the others), similar to the following:

Available options:

1. Add a project by clicking “Start new”
2. Import projects from .grd () files by clicking “Import”

Each project can be shared with the other users by clicking “share” button on the right side of the project tab.

On the top right, there is a management panel with several options, allowing for instance to access to the GRADE Handbook, send feedback to the development team, change some basic options etc.

The system is designed to work both online and offline. When offline, the user can work his/her projects, and the tool will synchronize the data with the server upon the next connection to the Internet.

The Project screen

Project screen is the main view of the user project:

In the left side panel the user can access the project menu with the following options:

Area in the center of the screen is the workspace. This is where the user inputs all the data, creates summary of findings tables and more.

The toolbar above it allows:

• Management of explanations/footnotes (Explanation button)
• Context-sensitive description of a particular field in tables (Help button)
• Selection of the table view (eye icon)
• Importing and exporting files
• Table printout

Tasks

Tasks screen is a basic module for event management.

It features a basic Calendar functionality, allowing to add project- or team-related tasks or events to the calendar and modify them.

Team

Team screen allows management of the project team members.

It has two different views:

1. Members - allows adding people to the project team, by their names and email addresses:

2. Conflicts of interest - this module enables the user to run and manage the Conflict of Interest collection process. Project moderator is able to select the appropriate CoI form and send it to the team members with one click. The answers are traced, collected and presented for the whole team and for each team member individually.

Scope

Scope is a complex question generation module. It helps a team to create, brainstorm and select the best questions to work on, in a 7-step process. All the communication throughout the brainstorming and voting process is automatically distributed by email to team members in a form of external web pages. All data is gathered back in the tool and presented to the project moderator.

The steps in the question generation process are:

1. Initial draft - the first step of the process; initial questions can be added here
2. Brainstorming - team members receive forms via email and can add and submit their proposed questions
3. Completed list - after questions from brainstorming are collected, the moderator reviews them and is able to add or modify the items
4. Prioritizing - users are asked by email form to prioritize the questions gathered during brainstorming
5. Final list for approval – the final list is shaped in this step
6. Approved list – the team members approve the question list
7. Finished – the questions that are marked as important are transferred automatically to the Comparisons module.

Document sections

This module provides a WYSIWYG (“what you see is what you get”) editor for creation of the final guideline document. The editor automatically fills the contents of the guideline document basing on the data from the question list, summary of evidence tables, and the recommendations, allowing the user to fill the additional descriptions etc.

The data can also be exported to an external editor (e.g. Word) if this is the user's preference.

Comparisons

Comparisons is a main and the most complex content module, enabling the users to create and author questions, outcomes, fill or import summary of findings tables and define the recommendations.  The module may be used with or without the question and outcome generation modules described earlier. Many users, who already have an established process and tools of their choice start with the “comparisons”, importing the data e.g. from RevMan or entering the list of questions manually.

The users are able to create management or diagnostic questions:

Within a question the users are able to choose between several different table views to input data. Users are also able to import data from other applications (RevMan 5 and former GRADEpro 3.0).

The fields in tables are editable in-place (WYSIWYG paradigm) and the system automatically computes values of relevant calculated  fields (e.g. assessment of the quality of the evidence).

It is possible to annotate the fields in the tables with appropriate footnotes by using “Explanations” button in the top toolbar. Also, the application will force the user to provide explanations in fields, where they are expected/necessary (it can be disabled in options).

After all the tables are filled, the user can move to fully editable Recommendations table, to provide all final decisions to the guideline. This can be done – if needed – during the session of the guideline panel.

Templates for different categories of decisions

We have developed different templates for EtD frameworks to help groups of people (panels) making healthcare recommendations or decisions move from evidence to decisions. The different templates that are tailored for different types of recommendations and decisions, taken from different perspectives (e.g., individual, population). These include:

• clinical practice recommendations about preventive, therapeutic, or rehabilitative interventions;
• decisions made on behalf of a population about which clinical interventions are paid for by a third party (coverage decisions);
• clinical practice recommendations about tests used for diagnostic, screening, monitoring or other purposes (tests), when there is no direct evidence of the effects of the diagnostic intervention on patient outcomes resulting from management decisions based on the test results;
• recommendations and decisions made on behalf of a populations about health system and public health interventions (options).

Most of the criteria are similar in the different templates. All include questions about whether the problem is a priority, the magnitude of the desirable and undesirable effects, the certainty of the evidence, consideration of how patients (or others affected, such as carers) value the main outcomes, the balance between desirable and undesirable effects, resource use, acceptability and feasibility. In addition, questions from a population perspective all include consideration of impacts on equity. For templates regarding tests when there is no direct evidence of the impact of alternative testing strategies on important outcomes, additional criteria are required. These include consideration of test accuracy, and the certainty of the different types of evidence used to inform judgments about the desirable and undesirable effects of a test (including direct effects, such as adverse effects from invasive tests, and indirect effects, resulting from management decisions based on the test results).

The iEtD includes standard templates for:

• Clinical recommendation – individual patient perspective – This template is for recommendations (about preventive, therapeutic, or rehabilitative interventions) for individual patients and clinicians made from the perspective of individual patients.
• Clinical recommendation – population perspective – This template is for recommendations for individualpatients and clinicians from the perspective of the population that is affected (e.g. a societal, government, department of health or health insurer's or hospital's perspective). These recommendations may also be targeted at policymakers or managers.
• Coverage decision – This template is for groups of people who make decisions about which clinical interventions are paid for (covered) by health insurance, a publicly funded healthcare system or a health maintenance organisation.
• Test - clinical recommendation - individual perspective – This template is for recommendations about tests when there is no direct evidence of the effects of the diagnostic intervention on patient outcomes resulting from management decisions based on the test results. These recommendations are for individual patients and clinicians from the perspective of the individual patient.
• Test - clinical recommendation - population perspective – This template is for recommendations about diagnostic interventions when there is no direct evidence of the effects of the diagnostic intervention on patient outcomes resulting from management decisions based on the test results. These recommendations are for individual patients and clinicians from the perspective of the population that is affected (e.g. a societal, government, department of health or health insurer or hospital's perspective). These recommendations may also be targeted at policymakers or managers.
• Test - coverage decision – This template is for groups of people who make decisions about which tests are paid for (covered) by health insurance, a publicly funded healthcare system or a health maintenance organisation
• Health system and public health recommendation – This template is for recommendations about health system or public health interventions (options) where decisions must be made by a group of people on behalf of the population that will be affected by those decisions. These recommendations are for policymakers, managers and stakeholders (those affected) with an interest in these decisions.
• Health system and public health decision – This template is designed to help decision-making by policymakers and managers (or the people advising them), who are responsible for making decisions about health system or public health interventions (options) that are targeted at a population (or that affect the choices available to individuals).

The EtD frameworks are flexible. Organisations may elect to modify these templates using the Administration>Templates module in the left panel.

They can modify the criteria and presentation elements to ensure that the framework is fit for their purpose. For example, guideline developers may have assessed the priority of problems prior to making recommendations and therefore might elect not to include this as a criterion.

Dissemination

This feature enables the user to publish her work.

The two options available currently are publication as a mobile app and to the Database of Evidence Profiles (DBEP).

The publication of a mobile app can be tested in a simulator or on a mobile phone/tablet. However, the publication of the app to the AppStore and Google Play requires assistance of the site Operator (Evidence Prime) and is a paid service, due to the requirements of Apple and Google and complexity of the process.

Video Tutorials

More in-depth video tutorials for the GRADEpro can be found under the links below:

http://www.youtube.com/watch?v=jFYHuB41gjQ

http://www.youtube.com/watch?v=EplFwVuhNwY

http://www.youtube.com/watch?v=00YzqsZ5ecs

http://www.youtube.com/watch?v=tQ6YrmpM8Jg

http://www.youtube.com/watch?v=tTIlVDTR0Fs

http://www.youtube.com/watch?v=uuf5s8phGPY

http://www.youtube.com/watch?v=PaTFjcwIUJs

http://www.youtube.com/watch?v=EhOirbUXhQs

http://www.youtube.com/watch?v=FYqp8H1oSMw

http://www.youtube.com/watch?v=XDvPbyOa8NE

http://www.youtube.com/watch?v=j64yHhKZZjQ

http://www.youtube.com/watch?v=2WQGpgduzxc

http://www.youtube.com/watch?v=NdBfDMIuJ6I

http://www.youtube.com/watch?v=HPCMp-x-7IY

About Evidence Prime

Evidence Prime is a McMaster University spinoff company creating innovative IT tools for Evidence-Base Health Care professionals.

Evidence Prime is also a designer, developer and sponsor of the GRADEpro GDT, software based on the intellectual work of the GRADE Working Group and a successor to the former Windows-based GREADEpro 3.0 software.

Evidence Prime offers many of its tools free of charge for the academic and non-commercial use, and actively participates in various efforts to develop standards supporting the exchange and sharing of the raw and processed information created by the Evidence-Based Health Care groups and institutions worldwide.