Oracle

Engineering Oracle Data Guard Redo Transport for Predictable Commit Latency

Pinterest LinkedIn Tumblr

Protection mode is also a latency architecture

Oracle Data Guard protection mode is often selected from an RPO matrix: Maximum Performance permits asynchronous transport, Maximum Availability targets zero data loss while allowing the primary to continue after a standby failure, and Maximum Protection prioritizes zero data loss even when that requires stopping the primary. That classification is necessary, but incomplete. With synchronous transport, the choice becomes part of every foreground commit.

A commit may depend on local redo flush, network transit, remote receipt and, depending on the transport mode, persistence at the destination. Storage latency, packet loss, listener behavior, routing convergence, encryption overhead, and standby redo log performance therefore become database response-time dependencies. A configuration can satisfy its nominal protection mode while producing unacceptable latency during partial failures.

The engineering objective is not simply to enable SYNC. It is to define which acknowledgment constitutes durability, how long foreground work may wait, when protection may degrade, and how applications behave during that transition.

Understand the acknowledgment boundary

Data Guard offers three relevant transport behaviors:

  • ASYNC decouples foreground commits from remote acknowledgment. It minimizes WAN impact but permits data loss equal to redo not yet received by the standby.
  • SYNC AFFIRM acknowledges after redo is written to the standby redo log. This provides the strongest remote durability boundary but exposes commits to remote storage latency.
  • FASTSYNC NOAFFIRM acknowledges receipt without waiting for the remote redo write to complete. It reduces latency but introduces a loss window if the primary and receiving standby fail before buffered redo becomes durable.

FASTSYNC should not be described operationally as equivalent to durable synchronous replication. It is a deliberate trade: lower commit latency in exchange for dependence on volatile state at the receiver. Architecture documents, recovery contracts, and failover procedures must use the actual acknowledgment semantics rather than the protection-mode label alone.

Measure the complete path. A low ICMP round-trip time does not prove that LGWR can obtain timely acknowledgments. Redo traverses Oracle Net, server scheduling, receiver processes, and the standby redo log storage path. Test with representative commit concurrency and redo volume, not only network probes.

NET_TIMEOUT defines the foreground failure budget

For synchronous transport, NET_TIMEOUT controls how long LGWR waits for an acknowledgment before declaring the destination failed. It is therefore an application-visible timeout, not merely a Data Guard tuning parameter. A value that is too high can suspend commits for an operationally unacceptable period. A value that is too low can convert harmless latency spikes into destination failures and protection-mode transitions.

A broker-managed configuration might use:

DGMGRL> EDIT DATABASE 'DRDB' SET PROPERTY LogXptMode='SYNC';
DGMGRL> EDIT DATABASE 'DRDB' SET PROPERTY NetTimeout=10;
DGMGRL> EDIT DATABASE 'DRDB' SET PROPERTY ReopenSecs=15;
DGMGRL> EDIT DATABASE 'DRDB' SET PROPERTY MaxFailure=3;

Do not copy these numbers without testing. Select NET_TIMEOUT from observed high-percentile acknowledgment latency plus a defensible margin, then compare it with connection-pool, transaction, load-balancer, and user-request timeouts. If an application abandons a request after five seconds while LGWR can wait ten seconds, the database may commit after the client reports failure. Retry logic can then create duplicate business operations unless transactions are idempotent.

REOPEN controls the minimum interval before retrying a failed destination. MAX_FAILURE limits retry behavior and participates in alternate-destination handling. Aggressive retries can amplify an unstable network condition; slow retries unnecessarily extend degraded protection. Oracle documents these properties as redo transport failure-policy controls, and their interaction should be exercised rather than reviewed only as static configuration.

Standby redo logs are performance infrastructure

SYNC AFFIRM makes standby redo log storage part of the commit path. Place standby redo logs on storage engineered for sustained low write latency and predictable tail behavior. Average latency is misleading: periodic array destaging, SAN congestion, multipath failover, thin-provisioning expansion, or cloud storage throttling can dominate commit response time.

Each redo thread requires correctly sized standby redo logs, normally with at least one more standby redo log group than the number of online redo log groups for that thread. RAC increases the importance of thread-by-thread validation. Missing groups for one thread can remain unnoticed until an instance or role transition exercises that path.

SELECT thread#, group#, bytes/1024/1024 AS mb, status
FROM v$standby_log
ORDER BY thread#, group#;

SELECT thread#, COUNT(*) AS online_groups,
       MIN(bytes)/1024/1024 AS min_mb,
       MAX(bytes)/1024/1024 AS max_mb
FROM v$log
GROUP BY thread#
ORDER BY thread#;

Size mismatches and excessive log switches create operational pressure during peaks. Very large logs, however, can lengthen some archive and recovery workflows. Select sizes from redo generation characteristics, recovery objectives, and switch frequency rather than a universal rule.

Design for partial failures, not only complete outages

The difficult failures are asymmetric. The primary may reach the standby listener while redo acknowledgments are delayed. One RAC instance may use a different route. DNS may return an unreachable address. A firewall may preserve established sessions but reject reconnections. SAN latency may affect only the standby redo log disk group. Monitoring that checks merely whether both databases are open or mounted will miss these conditions.

In Maximum Availability, loss of the synchronous destination can eventually allow the primary to continue without zero-data-loss protection. That behavior preserves availability but changes the recovery guarantee. Treat the degradation as a high-severity state transition. It should page an operator and be visible to incident automation, not disappear inside a generic broker warning.

Alternate destinations and far sync instances can reduce dependency on a distant standby. They also add routing state and failure-policy complexity. An alternate that has never received production-scale redo is not a credible protection path. Validate its network route, authentication, standby redo logs, capacity, and transition timing. If RedoRoutes or alternate destinations are used, document whether transport automatically falls back when the preferred destination returns.

Monitor transport as a commit dependency

Broker status is necessary but insufficient. Collect database, operating-system, network, and storage signals with aligned timestamps. At minimum, monitor destination status and errors, transport and apply lag, redo generation rate, standby redo log activity, process health, log-switch frequency, and commit latency.

SELECT dest_id, status, target, destination,
       transmit_mode, affirm, net_timeout, reopen_secs, error
FROM v$archive_dest
WHERE target = 'STANDBY';

SELECT name, value, unit, time_computed, datum_time
FROM v$dataguard_stats
WHERE name IN ('transport lag', 'apply lag',
               'apply finish time', 'estimated startup time');

SELECT event, total_waits,
       time_waited_micro/1000000 AS waited_seconds
FROM v$system_event
WHERE event IN ('log file sync', 'log file parallel write')
ORDER BY waited_seconds DESC;

Interpret transport lag carefully. A zero or small lag does not prove that synchronous acknowledgments are fast; it only describes redo position at the measurement point. Correlate increases in log file sync with LGWR write latency, Data Guard transport waits, network retransmissions, and remote storage latency. On RAC, retain instance-level dimensions because aggregate metrics can hide one impaired thread or node.

Also alert on stale measurements. The DATUM_TIME value helps distinguish a genuine lag value from a standby that has stopped updating its statistics. A dashboard showing yesterday’s zero lag is worse than an explicit unknown state.

Validate role transitions as application events

A technically successful switchover can still be an application outage. Before a planned transition, validate broker health, network connectivity, static connect identifiers, SPFILE differences, standby redo logs, temporary files, and service placement. Useful broker checks include:

DGMGRL> SHOW CONFIGURATION VERBOSE;
DGMGRL> VALIDATE DATABASE VERBOSE 'DRDB';
DGMGRL> VALIDATE NETWORK CONFIGURATION FOR ALL;
DGMGRL> VALIDATE STATIC CONNECT IDENTIFIER FOR ALL;
DGMGRL> VALIDATE DATABASE SPFILE 'DRDB';

Oracle’s broker documentation notes that network validation checks connectivity using configured connect identifiers, while static-connect validation verifies the service required for instance restart. These tests identify configuration defects, but they do not replace an end-to-end exercise under load.

Test service relocation, connection draining, DNS or SCAN behavior, pool reconnection, transaction retry, and read/write routing. Confirm that clients cannot write to an old endpoint and that connection pools do not retain unusable sessions longer than the recovery objective. Record commit latency before, during, and after the transition.

Engineer and rehearse the degradation policy

The final design decision is organizational as much as technical: when synchronous protection is unavailable, should production continue? Maximum Availability answers that at the database layer, but the business must approve the resulting temporary RPO exposure. Some systems should continue; others should reject new writes or enter a controlled mode.

Encode the decision in runbooks and automation with safeguards. Define who can reinstate transport, whether a failover is permitted when redo gaps exist, how data-loss exposure is calculated, and what evidence is required before returning to normal status. Rehearse WAN loss, standby storage latency, listener failure, RAC node loss, and alternate-route activation.

A robust Data Guard design makes commit behavior predictable in both healthy and degraded states. Protection mode establishes the intent; transport timeouts, acknowledgment semantics, standby storage, application retry behavior, and tested operational policy determine whether that intent survives production failure conditions.

Write A Comment