Mindtools.io Scoring
Each program is evaluated for its quality, source credibility, and explanation of privacy. These ratings are described below.
Program Quality
Assesses the program’s general potential to benefit its target audience in real-world use from user experience perspective. It does so by summarizing Enlight quality assessment ratings (Usability, Visual Design, User Engagement, Content, Therapeutic Persuasiveness, and Therapeutic Alliance). More about our formula can be found in our scientific approach page.
Interpretation – 1-1.99: Very Poor; 2-2.99: Poor; 3-3.49: Fair; 3.5-5: Good
Source Credibility
Assesses the degree to which the product/developers can be trusted. Comprised of a summation of 5 criteria ratings as described below. Complete information about these criteria can be found here.
Interpretation – 1: Very Poor; 2: Poor; 3, 4: Fair; 5: Good; 6, 7: Very Good; 8: Excellent
- Owners’ Credibility (Yes =1); Does the app come from a legitimate source?
- Maintenance (Yes =1). Frequency of update, Maintenance of site. Last update – less than 6 months.
- Strong Advisory Support
2= A leading expert is part of the team OR more than one leading expert are part of its advisory board (e.g., associate professors who specifically develop these kinds of products in their career)
1= Clinical and design experts are part of the team/advisory board (e.g., junior researchers in the field, certified clinicians with experience in the field)
- Third-Party Endorsement (Yes =2). Product has been verified, given a good review, and endorsed by a legitimate/reliable source (e.g., APA; FDA; SAMSHA; NIH; NHS in the UK; NICE in the UK).
- Evidence of Successful Implementation.
Yes (=1) if one of the following is true:
– Mobile: more than 100,000 downloads. Website: monthly average > 50,000 unique users.
– Mobile: Between 20,000-100,000 downloads; Website > 10,000 unique monthly users. AND high user engagement with low retention rates examined by a third party.
– Implemented within: health system under usual care OR large group of clinicians (>1,000) officially refers patients to utilize it.
– Over 1,000 reviews > 4.0
Privacy Explanation
| 0 | Transparent |
| 1 | Almost Transparent |
| 2 | Questionable Transparency |
| 3+ | Not Transparent |
We examine 8 different criteria (see below) to assess which actions are taken to enable users to understand how well their privacy is maintained by the program. The score regards user ability to make educated decision about privacy matters and reflects the time of our review. Sum items score range 0–8 which are then coded into 4 levels of transparency as presented above. The smaller the score – the better privacy matters are explained through the program.
| No. | Item | Yes | N / A | No or can’t tell |
| Terms of Use | ||||
| 1 | The system informs users of the data journey in detail so they understand all sources of data exposure (and risks if their device or app are not password-protected). This includes data stored on servers and on the device | 0 | 0 | 1 |
| 2 | The system notifies users how their personal identifiable information will be kept confidential and secured. | 0 | 0 | 1 |
| 3 | The system notifies users about how gathered data may be used (e.g., for commercial reasons). | 0 | 0 | 1 |
| 4 | For programs explicitly designed to be used by minors, the system includes a section requiring the approval/supervision of a legal guardian. | 0 | 0 | 1 |
| 5 | The system explicitly tunnels users through the terms of use (privacy/data wise, including items #1-#3, and #4 [if applicable]) before program utilization. In cases in which all other items are N/A, the system generally states that it does not collect any data, identifications, etc. | 0 | 0 | 1 |
| Systems with In-House Social Platforms | ||||
| 6 | The system enables users to keep identifiers private (and this is the default setting). | 0 | 0 | 1 |
| 7 | It is apparent when information will be seen by other users/members even if data do not contain identifiers (e.g., when they are in a particular zone where data are not kept private). | 0 | 0 | 1 |
| 8 | The system warns users about providing private identifiable information (e.g., name, health information, home address) to other users on the platform. | 0 | 0 | 1 |
Subscribe to Amit Baumel’s newsletter on emotional wellbeing in the technological era.