Between 15:23 UTC and 22:07 UTC on December 5th, some customers in the EU region may have experienced errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time data. To prevent our customers from receiving a high volume of false positive alert notifications, we temporarily suppressed notifications for affected data intermittently during this timeframe.
We have investigated and resolved this service interruption and impacted services have returned to normal operations.
Posted Dec 05, 2023 - 22:48 UTC
Monitoring
Our teams are continuing to investigate issues with data ingest for multiple data types. Some customers in the EU region may experience intermittent errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time data.
We are seeing recovery of affected services at this time and have disabled alert notification suppression as a result. We're monitoring as services recover and will provide an additional update once we've fully validated that all affected services have returned to normal operation.
Posted Dec 05, 2023 - 22:08 UTC
Update
Our teams are continuing to investigate issues with data ingest for multiple data types. Some customers in the EU region may experience intermittent errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time data. To prevent our customers from receiving a high volume of false positive alert notifications, we have temporarily suppressed notifications for affected data.
Updates for this interruption are suspended for one hour pending significant change in status as our teams continue their investigation.
Posted Dec 05, 2023 - 21:28 UTC
Update
Customers may continue to see improvement of impacted services as we work to investigate issues impacting ingested infrastructure data in the EU region. To prevent our customers from receiving a high volume of false positive alert notifications, notifications for impacted data will remain temporarily suppressed. Updates for this interruption are suspended for one hour pending significant change in status as our teams continue their investigation.
Posted Dec 05, 2023 - 20:28 UTC
Update
We are continuing to investigate an issue with data ingest for infrastructure data. Updates for this interruption are suspended for one hour pending significant change in status as our teams continue their investigation.
Some customers in the EU region may continue to experience errors when sending infrastructure data to New Relic, issues viewing real time infrastructure data, and delayed or missing alert notifications related to real time infrastructure data. To prevent our customers from receiving a high volume of false positive alert notifications, we have temporarily suppressed notifications for affected data.
Posted Dec 05, 2023 - 19:15 UTC
Update
We are continuing to investigate an issue with data ingest for infrastructure data. Some customers in the EU region may experience errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time infrastructure data. To prevent our customers from receiving a high volume of false positive alert notifications, we have temporarily suppressed notifications for affected data.
Customers may continue to see improvement as we work to recover impacted services.
Posted Dec 05, 2023 - 18:42 UTC
Update
We are continuing to investigate an issue with data ingest for infrastructure data. Some customers in the EU region may experience errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time infrastructure data. To prevent our customers from receiving a high volume of false positive alert notifications, we have temporarily suppressed notifications for affected data.
Posted Dec 05, 2023 - 18:00 UTC
Update
We are continuing to investigate an issue with data ingest for multiple data types. Some customers in the EU region may experience errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time data. To prevent our customers from receiving a high volume of false positive alert notifications, we have temporarily suppressed notifications for affected data.
Infrastructure and metric data remain impacted, but customers may now see recovery for all other ingested data types.
Posted Dec 05, 2023 - 17:28 UTC
Update
We are continuing to investigate an issue with data ingest for multiple data types. Some customers in the EU region may experience errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time data. To prevent our customers from receiving a high volume of false positive alert notifications, we have temporarily suppressed notifications for affected data.
Some customers may begin seeing the recovery of impacted services as we continue working to resolve this interruption.
Posted Dec 05, 2023 - 17:10 UTC
Update
We are continuing to investigate an issue with data ingest for multiple data types. Some customers in the EU region may experience errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time data. To prevent our customers from receiving a high volume of false positive alert notifications, we have temporarily suppressed notifications for affected data.
Posted Dec 05, 2023 - 16:40 UTC
Update
We are investigating an issue with data ingest for multiple data types. Some customers in the EU region may experience errors when sending data to New Relic, issues viewing real time data, and delayed or missing alert notifications related to real time data.
Posted Dec 05, 2023 - 16:09 UTC
Investigating
We are investigating a potential service interruption that may impact some customers. Please continue to monitor this page for updates.
Posted Dec 05, 2023 - 16:03 UTC
This incident affected: Alert Notifications : Europe (APM : Europe, Browser : Europe, Infrastructure : Europe, Logs : Europe, Mobile : Europe, NRQL : Europe, Serverless : Europe, Synthetics : Europe) and Data Ingest : Europe (APM Agent : Europe, Browser Agent : Europe, Event API : Europe, Incident Intelligence API : Europe, Infrastructure Agent : Europe, Log API : Europe, Metric API : Europe, Mobile Agent : Europe, OTLP API : Europe, Trace API : Europe, Synthetics : Europe).
