An honest Open Lines report: real response time and a monthly AI-assisted quality review
The built-in "First message response time" column counted the robot's auto-reply and showed "0 seconds". We built a report that counts the first live reply against the shift schedule, plus a monthly quality review with AI.
The problem
The head of support wants to see how operators respond to customers: how many requests came in, which were answered, how fast and how well. The data for this exists only in Bitrix24 Open Lines. But the built-in figures in the lines do not answer the main question: who actually replied and how quickly.
Before
- The "First message response time" column counted the robot's auto-reply. A live operator could answer an hour later, and the report showed "0 seconds". In a sample of 70 conversations we found this discrepancy in 10.
- The API does not return the conversation's operator: the bot is listed as responsible. The operator had to be reconstructed from service marks in the conversation.
- A conversation was credited to whoever took it last. One employee replied, another closed it, and the miss was recorded against the second.
- Requests outside a shift produced "0 seconds of waiting" and neatly inflated the averages. Operators work on different schedules, including evenings and past midnight.
- Quality was assessed manually against the criteria of the client's document: slow and uneven.
What we built
We assembled a set of scripts that read Open Lines correspondence in read-only mode.
- The operator of a conversation is determined by the first live reply that reached the customer. The bot, system messages, auto-replies and service sessions are not counted. We checked it against the portal export: 70 out of 70 for the single-employee report.
- Response time is counted in the operator's working hours according to their schedule, and days off do not count. A request outside a shift is counted from the start of the next shift. Calendar time is shown for reference.
- Three kinds of reports: for a single employee, for the whole team broken down by projects and by operators, and a list of conversations for the manager with unanswered requests. Service conversations are excluded so they do not inflate the "unanswered" figure.
- A monthly quality review against seven criteria: response time is calculated by a script, while the rest (answer quality, empathy, compliance with the contract and policies, communication standards, initiative, closing time) are assessed by AI according to the rules and scales from the client's document. For every employee up to 5 closed conversations from different weeks are picked by random sampling. Disputed cases are marked "needs review" and go to a human.
- The result is assembled into one Excel workbook: conversations, statistics by employee, project and operator, anomalies, and a "how we counted" explanation. Links lead to the conversations themselves, and months are added as rows.
Results
- The manager sees the real first-response time instead of artificial zeros and understands who actually replied.
- The report is assembled by scripts rather than by hand: the first run of the conversation list for a month takes about half an hour, a repeat run about a minute.
- The quality review has become regular and comparable from month to month, and changes to the methodology (columns in hours, dropping the punctuation check) are made once in the scripts.
- The manager also sees the weak spots honestly: the night tail of requests, unanswered conversations, customers who came back repeatedly.
The client's name is withheld under NDA. Real names and details are shared at the brief.
A similar task?
Describe it briefly — within 1–2 days we'll come back with an analysis and an hourly estimate. If a packaged solution fits, we'll say so honestly.