How we count impressions, responses and rates
In this article, you learn when PollHero counts an impression, a response or a completion and how the metrics in the reporting are calculated. This helps you interpret the numbers correctly and compare them with other tools such as your web analytics tool.
Key terms
| Term | Meaning |
|---|---|
| Impression (survey) | A survey was shown to a website visitor. This starts a participation. |
| Participation | One run of a person through the survey, from the first display to completion or drop-off. |
| Impression (question) | A single question was shown within a participation. |
| Response | An answered question. There is at most one response per participation and question. |
| Participant | A participation in which at least one question was answered. |
| Completion | A participation in which the survey was filled out to the end. |
PollHero counts participations, not people. If a person is allowed to fill out the survey several times (setting Always under Users see the survey), each participation counts separately.
When does an impression count?
An impression only counts once the survey actually becomes visible. When that happens depends on the survey type:
| Survey type | The impression counts … |
|---|---|
| Onsite survey, open | … as soon as the survey is shown, i.e. when the condition under The survey is displayed is met (e.g. after X seconds, at a scroll depth or on exit intent). |
| Onsite survey, started minimized | … only when the visitor clicks the badge or speech bubble and opens the survey. The badge alone does not count. |
| Embedded survey | … only when the container with the survey is scrolled into the visible area of the page. If the survey is embedded further down and the visitor never scrolls there, no impression is counted. |
| Stand-Alone survey | … when the survey page is opened and at least 10 % of the survey is visible. |
| Embedded Mail survey | … only when recipients click an answer in the email. Opening the email is not measured. |
Important:
- If a visitor reloads the page or moves to another page before finishing the survey, their participation continues. No new impression is created.
- A new impression is only created when a new participation is allowed to start, for example with the setting Always after a completion.
- Impressions are only counted for visitors who are allowed to see the survey according to your targeting and the settings under Users see the survey.
A question counts as shown as soon as at least 10 % of it is visible. Within one participation, each question counts at most once, even if the visitor goes back.
When does a response count?
Each response is saved immediately as soon as the visitor moves on to the next question, for example with Next or by clicking a rating. If someone drops off in the middle of the survey, all responses given up to that point are kept and appear in the reporting.
- If someone changes a response, the old one is overwritten. Only the latest response counts.
- Anyone who passes a question with Skip gives no response. The question then only counts as shown.
- If an optional question is submitted empty, it does not count as a response.
When is a survey considered completed?
A participation counts as completed when
- the last question has been answered. This happens before the Thank You Message appears, so the thank-you message does not need to be shown or clicked.
- logic ends the survey (close the survey or Show Thank You Message).
- the visitor closes the survey after answering all mandatory questions up to that point.
A participation is not considered completed if not a single question has been answered yet or a mandatory question was left open.
The metrics in the reporting
Survey metrics
| Metric | Calculation |
|---|---|
| Total impressions | Number of participations, i.e. how often the survey was shown (see above) |
| Total participants | Number of participations with at least one response |
| Completion rate | Completions ÷ total participants × 100. The number of completions is shown in brackets. |
| Active since | Date on which the survey was started |
The completion rate refers to the participants, not the impressions. It shows how many of those who started the survey also finished it.
Metrics per question
| Metric | Calculation |
|---|---|
| Responses | Number of participations in which this question was answered. For Multi Select, each respondent counts once, no matter how many options they chose. |
| Impressions | Number of participations in which this question was shown |
| Response rate | Responses ÷ impressions of the question × 100 |
For the Engagement Trigger, clicks on the start button count as responses. Anyone who closes the survey there counts as a drop-off. For the Thank You Message, every interaction counts as a response: End, Custom button click and Survey closed. Both show the shares of their impressions. Anyone who does none of these appears under No interaction.
Values per question type
| Question type | Value shown | Calculation |
|---|---|---|
| Single Select | Share per option | Responses with this option ÷ all responses to the question × 100. Adds up to 100 %. |
| Multi Select | Share per option | Respondents who chose this option ÷ all responses to the question × 100. Usually adds up to more than 100 %, because several options are possible. |
| Rating Scale | ⌀ Rating, Median Rating | Average = sum of all ratings ÷ number of responses. Median = middle value of all ratings. |
| Net Promoter Score (NPS) | NPS | Share of Promoters (9–10) in % − share of Detractors (0–6) in %, rounded to a whole number. Range −100 to +100. |
| Customer Satisfaction Score (CSAT) | CSAT | Responses with 4 or 5 ÷ all responses × 100 |
| Customer Effort Score (CES) | ⌀ CES | Sum of all ratings ÷ number of responses |
| Emojis | Share per emoji | Responses with this emoji ÷ all responses × 100 |
| Ranking | Average position | Mean of the positions an option was ranked at |
| Matrix | Share per row and column | Responses with this rating in the row ÷ all responses to the question × 100 |
All values only include respondents who answered the question. In sentiment analysis, the percentages refer to all free text responses for which a sentiment was detected.
Trend over time
For NPS, Rating Scale, CSAT, Customer Effort Score and Emojis, the reporting shows the trend weekly or monthly. For NPS, CSAT and CES, the difference from the previous period is shown in points. Example: if the NPS rises from 20 to 30, the difference is +10 points.
Calculation example
An Onsite survey with three questions is shown 1,000 times in one week:
- 250 visitors answer at least the first question → Total participants: 250
- 200 of them answer all questions → Completion rate: 200 ÷ 250 × 100 = 80 %
- Question 2 is shown 240 times and answered 216 times → Response rate question 2: 216 ÷ 240 × 100 = 90 %
Relative to the impressions, 25 % of visitors took part (250 ÷ 1,000). The reporting does not show this value as a separate metric, but you can calculate it yourself from Total participants and Total impressions.
Date range, segments and time zone
- Date range: If you filter the reporting by a date range, impressions, responses and completions count on the day they occurred.
- Time zone: All dates in the reporting refer to UTC. A response at 0:30 a.m. German time is therefore still assigned to the previous day. Weeks run from Monday to Sunday.
- Segments: A segment selects users, for example all mobile users. All participations of these users are evaluated.
- Deleted questions: Responses to questions you deleted later remain visible in the reporting under Deleted questions and still count towards the participants.
- Reset results: If you reset the results, all impressions, responses and completions of the survey are deleted.
What is not counted
- Views of the survey preview, including via the preview link or QR code
- a minimized badge, as long as it is not clicked
- Embedded surveys whose container never becomes visible
- email opens for Embedded Mail surveys
- skipped questions (they only count as shown)
- visitors who don't see the survey because of targeting, sampling or the Users see the survey setting
Revenue and discount codes
If PollHero is connected to your shop, you can Activate revenue extrapolation in reporting in the settings of a question. This is evaluated for all question types except Short Free Text, Long Free Text, Mailing, Ranking and Matrix. The Revenue analysis & extrapolation table links responses with respondents' orders:
| Column | Calculation |
|---|---|
| Count, Percentage | Responses with this option and their share of all responses to the question |
| Revenue | Sum of all orders of the respondents who gave this response |
| AOV | Revenue ÷ number of these orders (average order value) |
| Extrapolated Orders | Orders of this response group, extrapolated to all orders of your project |
For the extrapolation, PollHero divides the number of all orders in your project by the number of orders of all respondents to this question. Orders and revenue of each response group are multiplied by this factor. This shows you how your responses would be distributed across your total revenue.
Good to know:
- All orders of the respondents are taken into account, including orders placed before they took part in the survey.
- Revenue, AOV and extrapolation always refer to the entire period. Date filters and segments only change Count and Percentage.
- Revenue values are updated every hour.
The Discount Code Summary shows how many respondents ordered with and without a discount code (Share of Responses), as well as revenue and AOV per discount code. It is updated once a day and also refers to the entire period.
When do the numbers appear?
| Data | Update |
|---|---|
| Impressions, responses, completions, response rates | immediately |
| Sentiment analysis and themes | after a few seconds to minutes |
| Revenue analysis & extrapolation | every hour |
| Discount Code Summary | once a day |
| AI summary (Key insights at a glance) | once a week (Mondays) |
Frequently asked questions
Why do I have fewer impressions than page views? An impression counts only once per participation, not per page view. In addition, only visitors who actually see the survey according to targeting and display settings are counted. For Onsite surveys started minimized and for Embedded surveys, the survey also has to be opened or become visible.
Why do the response rate and the completion rate differ? The response rate refers to a single question and its impressions. The completion rate refers to the whole survey and all participants. Anyone who drops off after the first question lowers the completion rate, but not the response rate of the first question.
Why does a question have more impressions than responses, even though it is mandatory? Even with mandatory questions, visitors can close the survey or leave the page. The question was then shown but not answered.
Do my tests in the preview count? No. The preview doesn't send any data. If you test a live survey directly on your website, however, it counts like any other participation. You can reset the results before the actual launch.