Reliable Operations Is About More Than Technology: Lessons from Data Centers for SRE and E

What data center operations taught me about following procedures, questioning assumptions, and building teams that can operate reliably under pressure.
Ricardo Acosta
Grow from Software Engineer to Engineering Manager with practical, actionable guidance
Get in touch

Recently, I completed Oracle’s Data Center Operations Foundations certification. I took it because I wanted to understand better how data center operations work and, more importantly, what the people doing that work deal with every day.

I work in cloud operations, managing teams responsible for infrastructure reliability, incident response, and customer-impacting issues. Although our work ultimately depends on physical infrastructure, I haven't worked directly on the data center operations side. I thought learning more about their processes could help me understand their challenges and improve how our teams collaborate.

I expected to learn about data center operations. What I didn't expect was how many of the principles would connect directly to the way I manage SRE and cloud operations teams.

Some of these lessons might seem obvious to experienced engineers. Investigate before acting. Follow the procedure. Communicate clearly. Stop when something doesn't look right.

But there is a difference between knowing these principles and applying them consistently, especially when a customer is waiting for an update, a production system is degraded, and several people are asking what we're doing to fix it.

That's where operational discipline really matters.

We Don't Need Heroes

One phrase my boss often uses has stayed with me: "We don't need heroes."

It captures something important about how we want our SREs to operate.

When an incident happens, there is often pressure to act immediately. An engineer may recognize a familiar symptom, remember how a similar problem was resolved, and feel confident that the same solution will work again.

Sometimes that instinct is correct. Sometimes it leads to a second incident.

Our expectation is straightforward: investigate first, don't assume anything, follow the standard operating procedure (SOP), and escalate when needed.

This doesn't mean engineers should stop thinking independently or follow instructions without understanding them. It means that technical confidence should not replace evidence, and urgency should not become an excuse to skip important steps.

Imagine that several database servers become unreachable. An engineer recognizes the symptoms from a previous network incident and suspects the same underlying problem. They already have an idea of which change might restore connectivity.

The temptation is to go directly to that solution.

But what if the underlying cause is different this time? What if the proposed change affects additional systems? What if another team has already made a change that alters the situation?

Before taking action, the engineer needs to establish what is actually happening, review the available evidence, understand the scope of the impact, and follow the applicable runbook.

If the procedure doesn't cover the situation, or the evidence points in a different direction, that's the time to ask for help or escalate.

A good SRE doesn't need to solve every problem alone. They need to recognize the problem, understand the available options, execute safely, and involve the right people when the situation exceeds their authority or experience.

That is what I think my boss means by not needing heroes.

We need engineers who can be trusted to make sound decisions under pressure, not people who feel compelled to prove they can fix everything themselves.

Make It Safe to Stop and Ask Questions

Following a procedure is only part of the equation. Engineers also need to feel comfortable saying when something doesn't make sense.

One principle that stood out to me in data center operations is the importance of stopping when something appears unsafe or unclear.

This applies well beyond the physical data center.

In SRE and cloud operations, an engineer might be working through a complex runbook, preparing a change, or responding to an incident. They may reach a step that doesn't match the current system state or encounter instructions that leave room for interpretation.

What happens next depends partly on the team's culture.

In a team where people feel pressured to demonstrate independence, an engineer might guess, improvise, or continue because they don't want to look inexperienced.

In a healthier environment, they can pause and ask for clarification.

That is one reason we have a daily standup with our SREs. It gives the team an opportunity to raise questions, discuss blockers, and identify situations that need additional attention.

Managers help remove organizational blockers. Senior SREs can provide technical guidance and clarify complex runbooks or processes. When a problem requires expertise or authorization beyond the team's immediate capabilities, we can identify the appropriate escalation path.

The meeting isn't a replacement for incident escalation or real-time communication. If a production issue requires immediate attention, nobody should wait until the next standup to raise it.

Instead, the standup provides a regular opportunity to surface questions before they become larger problems.

The important part is that asking for help is treated as a normal part of operating complex systems.

As managers, we cannot expect engineers to raise concerns if our reactions make them afraid to do so. We need to show that asking a good question is a sign of responsibility, not weakness.

I'd rather have an engineer stop and explain why a procedure doesn't seem right than discover later that they continued despite having doubts.

Experience Is Useful, but It Can Also Create Risk

One of the recurring challenges in operations is the assumption that a familiar problem must have a familiar solution.

An engineer has performed a particular task several times. They understand the procedure, know the expected outcome, and have successfully completed it in the past.

Eventually, the task begins to feel routine.

That familiarity can make the engineer faster and more confident. It can also make them less attentive to the details that matter.

We've had escalations almost every week because someone assumed that a solution that worked before would work again without sufficiently checking the current circumstances.

The problem isn't necessarily a lack of technical knowledge. Sometimes the engineer knows the procedure very well. The problem is treating previous success as proof that the current situation is identical.

Production environments are rarely that predictable.

The same symptoms can have different causes. A system may have changed since the previous execution. Dependencies may be in a different state. A procedure that is safe under one set of conditions may have different consequences under another.

This is why following an SOP matters even when you have performed the task many times.

A runbook isn't just a document for inexperienced engineers. It is a mechanism for making execution more consistent and ensuring that important checks aren't skipped simply because someone feels comfortable with the task.

Before acting, confirm the current state, validate the assumptions behind the procedure, and make sure the intended action is appropriate for the situation.

If the procedure requires a pre-check, perform it. If the system state doesn't match the documented expectations, don't force the process forward just to complete the task.

And if the runbook is incomplete or outdated, raise that issue so the procedure can be improved.

The goal isn't to eliminate judgment. It is to apply judgment before acting, rather than using it as a justification for skipping safeguards.

Clear Communication Is Part of Reliability

In operations, communication is sometimes treated as something separate from the technical work. Engineers investigate and resolve the issue; someone else communicates the status.

In practice, the two are closely connected.

When multiple teams are involved in an incident, people need to understand what is happening, what has been verified, what remains uncertain, and what action is planned next.

Without that shared understanding, teams can work against each other, repeat investigations, or make decisions based on different assumptions.

This becomes especially important when a problem crosses organizational boundaries.

For example, an SRE team might identify symptoms that appear to originate in the network, while the data center operations team is investigating a physical infrastructure issue. Both teams may have useful evidence, but neither necessarily has the complete picture.

If the teams communicate only conclusions rather than observations, they can end up debating whose system is responsible instead of working together to establish the cause.

A better approach is to communicate what has been observed, what has been checked, what the evidence suggests, and what is still unknown.

There is an important difference between saying, "The network is broken," and saying, "These nodes became unreachable shortly after the network change. We have verified that the issue affects these systems, and we're investigating whether the change is related."

The second statement gives other teams something concrete to work with without presenting an unconfirmed theory as fact.

Clear communication also means knowing when to escalate. If an engineer reaches a point where the next action is unclear, the risk is increasing, or the issue exceeds their authority, that information needs to reach the appropriate people promptly.

Good communication doesn't require long explanations. It requires enough accurate information for the next person to understand the situation and make a useful decision.

During incidents, that can make a significant difference.

Understanding the Work of Adjacent Teams Makes Us Better Partners

This was the original reason I wanted to learn more about data center operations.

I work with systems that depend on physical infrastructure, but the teams responsible for that infrastructure have their own procedures, constraints, responsibilities, and risks.

It is easy to understand our own operational processes in detail while having only a general idea of what another team needs to do to support us.

That gap can create friction.

When an incident occurs, we naturally focus on the problem affecting our systems and the urgency of restoring service. The team on the other side may be working under a different set of constraints, following procedures designed to protect physical equipment, personnel, or other customers.

Understanding those constraints doesn't mean lowering our expectations or accepting unnecessary delays. It means approaching collaboration with a better understanding of what the other team is responsible for and why certain safeguards exist.

The same principle applies across engineering organizations.

SREs benefit from understanding how development teams deploy changes. Database engineers benefit from understanding the infrastructure their systems depend on. Managers benefit from understanding the operational realities faced by the people and teams they coordinate.

We don't need to become experts in every adjacent discipline. But learning enough about another team's work can help us ask better questions, provide better context, and coordinate more effectively when something goes wrong.

For me, the certification was a small step toward that understanding.

It also reinforced something I believe about engineering management: collaboration improves when we make an effort to understand the people and processes on which our own work depends.

The Manager's Role Is to Make Good Operational Behavior Easier

These principles aren't only responsibilities for individual engineers. Managers have an important role in creating the conditions that allow teams to apply them consistently.

We can tell engineers to follow procedures, investigate before acting, and escalate appropriately. But those expectations are harder to meet when runbooks are outdated, escalation paths are unclear, workloads leave no room for investigation, or engineers feel punished for asking questions.

Our job is to identify and remove those obstacles.

That might mean ensuring that runbooks are reviewed and maintained, clarifying who has authority to approve a particular action, or creating a regular forum where engineers can raise concerns.

It also means reinforcing the right behaviors when incidents happen.

If an engineer stops because the evidence doesn't support the planned action, that decision deserves support. If someone escalates a problem early because they recognize the limits of their experience, that should be seen as good judgment.

Of course, escalation shouldn't replace investigation. We still expect engineers to gather relevant information, explain what they've observed, and use the resources available to them.

The objective is to find the right balance between independent problem-solving and knowing when additional help is needed.

Over time, those expectations become part of how the team operates.

You don't build reliability simply by hiring technically strong engineers. You build it by creating an environment where people can apply their skills consistently, learn from mistakes, and work together without unnecessary friction.

Final Thoughts

Taking the Data Center Operations Foundations certification gave me an opportunity to learn more about a part of the infrastructure world I don't work in directly.

What stayed with me wasn't a single technical concept. It was how relevant the operational principles were to the work my own teams do every day.

Investigate before acting. Don't assume that a familiar symptom has a familiar cause. Follow the procedure, and speak up when something doesn't make sense. Communicate clearly, and involve the right people when needed.

These practices may not sound particularly exciting. They don't make for dramatic stories about an engineer who saved the day by solving an impossible problem alone.

But that is precisely the point.

Reliable operations shouldn't depend on someone being a hero at the right moment. They should depend on teams that investigate carefully, execute consistently, communicate clearly, and know when to stop and ask for help.

As engineering leaders, we have a responsibility to make that kind of work possible—and to recognize that disciplined execution is just as valuable as technical brilliance.

Ready to find the right
mentor for your goals?

Find out if MentorCruise is a good fit for you – fast, free, and no pressure.

Tell us about your goals

See how mentorship compares to other options

Preview your first month