Custom CEL rules temporarily deleted on cumulocity.com tenants

Follow

Issue Summary

A scheduled cleanup job deleted custom CEL rules in cumulocity.com tenants on April 30th, 2 a.m. CEST. Smart rules and custom CEL rules on other instances were not affected.

The CEL rules were successively restored from a backup by May 3rd. The CEL rules are restored in a deactivated state to not interfere with other rules introduced between April 30th and May 3rd. Customers are requested to activate the required CEL rules.

If you have deactivated support access to your tenant, the rules in your tenants may not have been restored. In this case, you can contact support through https://empower.softwareag.com  to activate support access and restore the rules.

Impact of the problem

During the absence of the CEL rules until their re-activation, processing of the rules has not been carried out (for example, custom measurement calculation, email sending).

Timeline

The table below gives the timeline of events.

Time (CEST)

Description

 

April 29th

Smart rules cleanup job installed as part of the Cumulocity 10.4.0 upgrade on cumulocity.com.

 

April 30th, 2 a.m.

Cleanup job deletes custom CEL rules.

 

April 30th, 5 p.m.

Cleanup job removed.

 

May, 2nd-3nd

Backup restored and CEL rules uploaded again.

 


Root Cause

The root cause was an incorrect implementation of a clean up algorithm in the Smart Rules microservice. The implementation bug was not found during specification, code review, automated testing, manual testing and pilot deployment.

Risk of recurrence

The algorithm was removed from the Smart Rules microservice.

Corrective and Preventative Measures

  1. Additional test case.
  2. Less aggressive reimplementation of the cleanup algorithm.

 

 

Have more questions? Submit a request

Comments