-
Notifications
You must be signed in to change notification settings - Fork 5
Description
During a large data load, the disk space ran out. DiSCO POSTs began to get a message saying "ID must not be null!" which points to this line of code. Because space ran out, we do not have error logs showing a stack trace.
Instead of just sending errors back on DiSCO POSTs, the service continued to try to replenish the ID Store by getting new ARKIDs from EZID. Though we have no log to prove this, our best guess is that because of the lack of disk space, the IDs could not be stored and so the process got stuck in a loop of retrieving new IDs.
The file "arkids.txt" shows the ARKIDs in the ID store once the disk space issue was resolved, but no new DiSCOs were posted to trigger replenishment of the store. The _updated timestamp retrieved from the EZID service has been added.
Log files are also available in our local Jira, but stop at the time the error started.
To fix this, we need to ensure that there is proper error handling when IDs are not written correctly to the store. The error should be detected before we get to "ID must not be null".