Modifications CRM

A nightly backup of cloud Bitrix24 and a watchdog for integration keys

For a cloud portal we built our own nightly copy of the data and a watchdog that checks integration keys every hour. The company learns about a breakage from a Telegram message, not from silence in the mailings.

Client International company, customer service on cloud Bitrix24
Industry Service platforms · customer support
Duration about 1 week
Type Cloud Bitrix24 backup and key monitoring
Once a day nightly portal backup
35 days backup retention depth
1 hour maximum time until a broken key is flagged

The problem

The company's entire customer service lives in cloud Bitrix24: employees, deals, contacts, lists, tasks, business process templates. The cloud is convenient, but the portal owner has no copy of the data of their own: if the portal becomes unavailable or is disabled, recovery would have to start from scratch. The second problem: several of the company's own services (mailings, reports, surveys, the connector) work around the portal, and they all reach it using inbound webhook keys.

Before

  • The company had no copy of its data: everything lived only in the cloud.
  • Webhook keys are tied to live employees. Bitrix24 may disable such keys when someone leaves and can limit their lifetime. A broken key shows itself only when delivery or reports quietly stop working.
  • Some of the services around the portal share a single request limit, so the backup must not get in the way of working processes.

What we built

Two small services on the company's server, both run on a systemd timer.

Nightly portal backup. It accesses the portal read-only and stores the data in a single database: employees and departments, groups with members, tasks, leads, deals of selected pipelines, contacts, companies, requisites, smart processes, reference lists, fields, lists with their items and business process templates. Every day it saves only the changes of the last 24 hours, and once a week it makes a full pass marking deleted records. Retention is 35 days, and a restore script assembles the database for any chosen date. It runs slowly and yields to the portal's limits, so it does not interfere with working services. A second copy of the snapshots is kept on a separate computer.

Key watchdog. Every hour it checks each webhook used by the company's services and the state of the connector application. If a key stops working, it writes to Telegram, repeats once a day until it is fixed, and reports when everything works again. It also checks where each line's Telegram bot webhook points, so you can see if someone's settings in the portal quietly took a bot back.

The watchdog takes keys straight from the services' configs and does not duplicate them. The backup has its own dedicated key, of whose permissions we use only reading.

Results

  • The company has its own copy of the portal data: the database can be assembled for any date within the last 35 days. Drive files, internal chats and the feed are not covered by the copy, and these limits are deliberate.
  • A backup failure does not go unnoticed either: the message arrives from the same bot.
  • A dead key is found within an hour, not when a client complains.
  • Regular summaries show that all keys and webhooks are alive, and they are part of the diagnostics after major work on the portal.
  • The watchdog and the backup account for the portal's shared request limit from the start and do not crowd out working processes.

The client's name is withheld under NDA. Real names and details are shared at the brief.

A similar task?

Describe it briefly — within 1–2 days we'll come back with an analysis and an hourly estimate. If a packaged solution fits, we'll say so honestly.

✈ Telegram