Confidentiality and the number of respondents

 

To ensure that answers cannot be traced back to individual respondents, Effectory has established various business rules to ensure confidentiality. These are based on the number of participants per group (i.e., per team) that respond to a survey. The rules we apply can be categorized as follows:

• Group size and response rate
• Confidentiality across groups
• Distribution of answers
• The number of responses to a question

 

Group size and response rate

To safeguard the confidentiality of participants' answers, we have established specific group size and response rate criteria for surveys. These are:

• Minimum group size: We only report group scores for groups with five or more participants. Organizations may opt to increase this minimum group size, typically between five and ten participants, with five being the absolute minimum. Groups falling below this requirement will not have their results reported separately, ensuring individual respondents remain untraceable.
  
  In the picture above, Design meets the minimum group size with six participants (of which five responded). Legal, however, doesn't meet the minimum group size even though all participants have responded.
  
  
• Minimum number of respondents: In addition to group size, we establish a minimum number of required respondents for each group, which can never be below five. If this requirement is not met, the group's results will not be reported at the specific group level but instead included in a parent group. You may opt to increase this threshold if needed.
  
  In the picture above, both L&D and Sales reached the minimum group size. However, only L&D reaches the minimum response rate and can be reported individually.
  
  
• Limited insights: The results dashboard will display limited insights (see picture below) for groups that do not meet the minimum response rate but have at least three respondents. These insights provide an indication of the group's performance without revealing the actual scores, maintaining a high level of confidentiality for respondents. For groups with less than three respondents, no insights are shown at all.
  
  In the picture above, Maintenance has enough participants, but only three responded. Its scores cannot be reported fully, but the dashboard will show limited insights for this group.

 

Confidentiality across groups

Sometimes, certain groups' scores may not be reported, although they have reached the minimum number of respondents, as doing so would endanger the confidentiality of participants from other groups on the same level.

Generally, three rules determine whether groups within the same parent group can be reported without compromising the confidentiality of participants:

• Any group must have at least five respondents to be reported individually.
• The sum of all non-reported respondents within a level must be at least five.
• The difference in reported respondents between two overlapping levels must be at least five.

If any of these rules are not met, exceptions may apply. Below are some common scenarios:

 

Scenario 1: Protecting an unreported group

Team Amsterdam is made up of two subgroups, Design and Legal. While Design meets the confidentiality threshold, reporting its score individually would make it possible to infer the score of Legal.

To maintain confidentiality, Design's score is not reported separately; instead, the combined scores of Design and Legal are reported at the parent level. Answers to open-ended questions are also grouped and only reported at the parent level (Team Amsterdam).

 

Scenario 2: Protecting an ungrouped respondent

Team Amsterdam consists of a single subgroup, Design. While Design meets the confidentiality requirements, there is one additional respondent in Team Amsterdam who is not part of Design—this could be, for example, Design's manager.

If Design’s results were reported separately from Team Amsterdam, it would become possible to infer the score of the single ungrouped respondent. To prevent this, Design's results are not reported separately, and all scores and answers to open-ended questions are aggregated at the parent level.

Note: This rule only applies when there is a single subgroup!

 

Scenario 3: Sufficient non-reported respondents

Team Amsterdam consists of three subgroups: Design, Legal, and Sales. While Design meets the confidentiality threshold, the other two groups do not. However, since the total number of non-reported respondents (Legal and Sales combined) is at least five, Design's results can be reported individually without compromising any respondent's confidentiality.

Legal's and Sales' scores and answers to open-ended questions are reported at the parent level combined with those of Design.

 

Scenario 4: Insufficient non-reported respondents (1)

In this scenario, Team Amsterdam again consists of three subgroups: Design, Legal, and Sales. However, unlike Scenario 3, the total number of non-reported respondents (Legal and Sales combined) is fewer than five.

Since revealing Design’s score would make it possible to infer other respondents’ scores, none of the subgroup results (Design, Legal, or Sales) are reported individually. Instead, all scores and answers to open-ended questions are aggregated at the parent level (Team Amsterdam) to maintain confidentiality.

 

Scenario 5: Insufficient non-reported respondents (2)

When more than one group at the same level meets the reporting criteria, it becomes increasingly difficult to deduce individual scores. In this case, even though the total number of non-reported respondents from Legal and Sales is fewer than five, Design and L&D can be reported separately without compromising any respondent's confidentiality, unlike Scenario 4. Legal's and Sales' scores are reported at the parent level combined with those of Design and L&D.

Open-ended answers from Design and L&D are reported individually as usual. However, Legal and Sales' answers cannot be reported individually or at the parent level to protect the non-reported participant’s confidentiality. Instead, they are included only at higher levels, where they can be combined with enough non-reported responses to make a group of five, ensuring confidentiality.

 

Scenario 6: Multiple reported groups

Team Amsterdam consists again of three subgroups: Design, L&D, and Sales. In this case, both Design and L&D meet the confidentiality threshold, while Sales does not. Like in Scenario 5, the total number of non-reported respondents is fewer than five. However, Design and L&D can be reported individually because more than one group at the same level meets the reporting criteria, making it more difficult to infer the score of the non-reported respondent. Sales' scores are reported at the parent level combined with those of Design, L&D.

Open-ended answers from Design and L&D are reported individually as usual. However, Sales’ answers cannot be reported individually or at the parent level to protect the non-reported participant's confidentiality. Instead, they are included only at higher levels, where they can be combined with enough non-reported responses to make a group of five, ensuring confidentiality.

 

Distribution of answers

The results dashboard shows the distribution of answers for most questions to provide additional insights into the range and frequency of responses.

To protect the confidentiality of participants' answers, My Effectory applies one of two approaches depending on the number of respondents in the group in question:

• More than 50 respondents: A 5-point scale is used.
• Fewer than 50 respondents: A 3-point scale is used.

 

The number of responses to a question

Effectory handles the number of responses per question by implementing a 5/5 threshold at the group level rather than on an individual question basis. This means that, without exception, all survey questions are always reported, even those that have gathered fewer than five responses.