Online Failure - Hidden Timer Issues

Recently encountered a serious issue online where our server was frantically sending heartbeat requests to another SDK service provider. After investigation, it turned out to be caused by an overlooked point related to timers, so marking it here.

Problem Situation

The current situation is that one service IP made over 4000 requests per minute. Actually, under normal circumstances, it should only report once every 5 minutes.

Analysis

The entire troubleshooting process was relatively simple/clear

1. After confirming, the heartbeat request is automatically triggered by the SDK itself when initializing instance objects. Our logic has no heartbeat setup or heartbeat calling logic, so it must be an initialization logic problem
   • The SDK class constructor has setInterval logic
2. Querying the program revealed there are currently 4 places initializing instance objects, with 1 place initializing at service startup, and the other 3 triggered during API calls. So if API calls are frequent enough, it would bring N timers
3. Checking production logs, the increased frequency of related API calls also correlates with data time points monitored by another service

OK, after identifying the problem, the solution was also clear - remove the logic of instantiating SDK objects during API calls. According to our actual situation, we should maintain a singleton and initialize it at service startup.

After fixing, re-monitoring showed it’s back to normal.

Summary

1. When JS functions finish executing, they destroy related objects but do not destroy timers. Therefore, when calling logic that starts timers, you must pay attention to cleanup logic.
2. As above, calling certain functions that contain hidden timer logic can be a pitfall. This needs extra attention.