All Systems Operational

About This Site

Subscribe to this page to stay up-to-date on the operational status of Baicells cloud-based systems.

CloudCore Portal Operational
90 days ago
99.96 % uptime
Today
Cloud OMC Operational
90 days ago
99.9 % uptime
Today
Cloud BOSS Operational
90 days ago
100.0 % uptime
Today
Cloud EPC Operational
90 days ago
100.0 % uptime
Today
US East MME Operational
90 days ago
100.0 % uptime
Today
US West MME Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Active eNodeBs
Fetching
Active CPEs
Fetching
Past Incidents
Feb 29, 2024

No incidents reported today.

Feb 28, 2024

No incidents reported.

Feb 27, 2024
Resolved - After monitoring the OMC for a few hours, no further issues have been discovered and the domain proxy service has been operating in a normal state. Marking the incident as resolved.

In summary, the Google SAS server was operating as expected and the outage was due to some maintenance made on the test servers last night which were mistakenly replicated to our live servers.

Feb 27, 15:39 CST
Monitoring - Root cause has been identified. Last night, our development team was prepping the test environment for the new CloudCore update we will be announcing soon. While updating the test environment, our development team mistakenly triggered grant relinquishes for some CBSDs on the commercial environment. This ultimately caused the heartbeat issues that were occurring this morning. We will be evaluating internally on how exactly this happened and will prevent this from occurring in the future.

As for now, any remaining heartbeats which receive 103 responseCode will automatically trigger a deregistration and reregistration. Will keep this incident open and continue to monitor to ensure all CBSDs are operating in a normal state.

Feb 27, 11:51 CST
Identified - The heartbeat errors due to 103 response code are due to the grant IDs being invalid. However, the grant IDs were never changed and the response codes are only found after the CPAS event this morning. We are still working with Google to understand the root cause.

The workaround for the time being would be to deregister and reregister the CBSDs. To manually perform this task please refer to the following steps:

1. Login to CloudCore
2. Navigate to Advance>SAS page
3. Click the checkbox next to each impacted CBSD
4. Click the "SAS OFF" button
5. Wait for CBSDs to indicate the SAS is disabled
6. Click the "SAS ON" button

Feb 27, 10:02 CST
Investigating - We are currently investigating an issue with some heartbeat messages erroring out with Google SAS with a 103 INVALID VALUE code. Due to the heartbeat errors, the impacted eNBs are disabling RF once the transmitExpire time triggers. This will impact all users who have CBRS equipment and are using Google SAS.
Feb 27, 08:38 CST
Feb 26, 2024

No incidents reported.

Feb 25, 2024

No incidents reported.

Feb 24, 2024

No incidents reported.

Feb 23, 2024

No incidents reported.

Feb 22, 2024

No incidents reported.

Feb 21, 2024

No incidents reported.

Feb 20, 2024

No incidents reported.

Feb 19, 2024

No incidents reported.

Feb 18, 2024

No incidents reported.

Feb 17, 2024

No incidents reported.

Feb 16, 2024

No incidents reported.

Feb 15, 2024

No incidents reported.