1. Home
  2. Results & Analytics
  3. Anonymity and hidden answers

Anonymity and hidden answers

The anonymity threshold

In order to protect the respondents’ integrity, reports are not available if the number of answers for a specific survey cycle in a team or group is lower than the anonymity threshold. This means that a team in which the total number of employees is lower than the anonymity threshold will not be able to get a separate report.

What is our anonymity threshold?

The minimum number of answers required to get a report is called the “anonymity threshold“. This threshold is the same across all teams and is chosen by your organization. If you are unsure about which threshold your organization uses, you are welcome to contact us.

Hidden answers

Hidden answers are answers submitted by a user that cannot be shown in a specific report due to anonymity reasons. When an answer is hidden in a report, it will not be included in the calculations of any of the results in that specific report. Hidden answers are however included when the response rate is calculated.

Hidden answers always originate from a team/group that has too few answers to obtain a report of their own, and when viewing a report on a higher level (where they would be expected to be included), they are hidden by the anonymity algorithm. This can occur even if the total number of answers in the report you are viewing is well above the anonymity threshold, if it would otherwise be possible to identify these answers by comparing different reports. 

Example 1

Imagine the example illustrated in Figure 1 below. John Doe has 3 direct reporting employees, of which 1 is a manager herself (User 2). Users 1 – 4 did respond to the survey on for this specific cycle, while Users 5 & 6 did not respond. The anonymity threshold for this organization is 3. In John Doe’s organization, there can be a maximum of 3 different reports (given that all employees respond to the survey):

  1. A report for John’s direct reporting team (3 employees, Users 1 – 3)
  2. A report for User 2’s team (3 employees, Users 4 – 6)
  3. A total report for John’s full organization (6 employees, Users 1 – 6)

Since all of John’s direct reporting employees responded to the survey, John’s team report (Report A) will be available (as the number of answers is equal to the anonymity threshold). Since there is only 1 answer in User 2’s team, she will not get a separate report (Report B) for this specific cycle. This means that in the report for John’s full organization (Report C), the answer from User 4 will be hidden to protect User 4’s anonymity. The reason for this is that it would otherwise be possible to identify the answers from User 4 by comparing reports A and C (the difference would be equal to User 4’s answers). In this case, Report A and Report C thus become identical.

Figure 1 – Illustration of example 1

When will hidden answers become visible?

Answers that originate from a team/group that has too few answers to obtain a report of their own will be shown in a report on a higher level when the total number of such answers is equal to or higher than the anonymity threshold. Once an answer is shown at a certain level, it will always be shown in reports on higher levels.

Example 2

Imagine the example illustrated in Figure 2 below. In this example, John Doe has 3 direct reporting employees, of which 2 are managers themselves (Users 1 & 3). Users 1 – 4 and 7 & 8 did respond to the survey for this specific cycle, while Users 5 & 6 did not respond. The anonymity threshold for this organization is 3. In John Doe’s organization, there can now be a maximum of 3 different reports (given that all employees respond to the survey):

  1. A report for John’s direct reporting team (3 employees, Users 1 – 3)
  2. A report for User 1’s team (3 employees, Users 4 – 6)
  3. A total report for John’s full organization (8 employees, Users 1 – 8)

Note: Since User 3’s team is only 2 employees, that team can never get a report of their own.

Since all of John’s direct reporting employees responded to the survey, John’s team report (Report A) will be available (as the number of answers is equal to the anonymity threshold). Since there is only 1 answer in User 1’s team, she will not get a separate report (Report B) for this specific cycle.

However, since there are now also 2 answers in User 3’s team that are not shown in a team report, there will be no hidden answers in the report for John’s full organization (Report C). The reason for this is that the total number of answers originating from a team with too few answers to obtain a report is now equal to the anonymity threshold (3). Because of this, it is no longer possible to identify a specific answer when comparing the different reports (by comparing Report A and C, it will only be possible to identify which 3 answers originate from User 1 and User 3’s teams, which is allowed given the anonymity threshold of 3).

Figure 2 – Illustration of example 2

Groups and hidden answers

In case a manager has divided his / her team into 2 or more groups, the individual groups’ reports will always take precedence over the manager’s total report when the reports are created. If, for example, a manager has 2 groups with 3 employees in each group, but only 2 employees participated in Group A (while all 3 employees in group B participated), Group B will receive a report of its own, while Group A will not. The responses from the employees in Group A will now be hidden in the manager’s total report, since it would otherwise be possible to compare Group B’s report with the total report, and thereby identify the 2 responses from Group A (which violates the anonymity threshold).

Updated on February 19, 2020

Was this article helpful?

Related articles