The ResourceManager's Two Faces
The ResourceManager runs two logically distinct components inside one daemon: the Scheduler, which allocates resources to running applications purely based on capacity constraints (it does no monitoring or restart of failed tasks), and the ApplicationsManager, which accepts job submissions, negotiates the first container to launch each application's ApplicationMaster, and restarts that AM container if it fails. Splitting these concerns keeps the Scheduler's hot path, handling thousands of heartbeats per second, free of the bookkeeping needed for application lifecycle management.
Cricket analogy: The Scheduler is like a stadium's ground-allocation clerk who only checks whether a slot is free, while the ApplicationsManager is like the tournament office that actually registers each team like Mumbai Indians for the season and reissues their fixture if a match is postponed.
The NodeManager on Every Worker Node
Every worker node runs one NodeManager, which registers with the RM at startup, advertises its total available memory and vcores (configured via yarn.nodemanager.resource.memory-mb and yarn.nodemanager.resource.cpu-vcores), and then manages the full lifecycle of each container assigned to it: launching the container's process, monitoring its actual resource usage against its granted limit using Linux cgroups, and killing it if it exceeds its memory ceiling. The NodeManager also runs auxiliary services, most notably the ShuffleHandler for MapReduce, which serves map output to reducers even after the producing container has exited.
Cricket analogy: Like each individual stadium's facilities office registering its total seating and pitch capacity with the tournament board, then physically ejecting any group like a corporate box that exceeds its booked headcount.
Heartbeats, Leases, and Failure Detection
The NodeManager sends a heartbeat to the ResourceManager roughly every second (configurable via yarn.resourcemanager.nodemanagers.heartbeat-interval-ms), reporting node health, container status changes, and available capacity; if the RM does not receive a heartbeat within yarn.nm.liveness-monitor.expiry-interval-ms (default 600 seconds), it marks the node LOST and reschedules its containers elsewhere. Because losing the ResourceManager itself would stop the whole cluster, production deployments run RM High Availability with an Active/Standby pair coordinated through ZooKeeper-based leader election, with application and scheduler state persisted to the RM state store for fast failover.
Cricket analogy: Like each fielder radioing their position to the captain every over, and the captain replacing a fielder who goes silent for too long with a substitute, YARN reschedules containers off a node that stops heartbeating.
<configuration>
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>rm1.cluster.internal</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>rm2.cluster.internal</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>zk1:2181,zk2:2181,zk3:2181</value>
</property>
</configuration>Setting yarn.nm.liveness-monitor.expiry-interval-ms too low in a cluster with occasional network blips will cause healthy NodeManagers to be marked LOST and their containers rescheduled unnecessarily, wasting work and adding cluster churn.
- The ResourceManager internally separates the Scheduler (pure resource allocation) from the ApplicationsManager (job lifecycle and AM restart).
- Every worker node runs one NodeManager, which registers its total memory and vcores and enforces container limits via cgroups.
- NodeManagers heartbeat to the RM roughly every second; missing heartbeats past the expiry interval mark a node LOST.
- A lost node's containers are rescheduled elsewhere, which can mean redoing in-flight work for non-checkpointed tasks.
- Production clusters run RM High Availability with an Active/Standby pair coordinated via ZooKeeper leader election.
- The ShuffleHandler is a NodeManager auxiliary service that lets reducers fetch map output even after the map container has exited.
Practice what you learned
1. Which ResourceManager sub-component restarts a failed ApplicationMaster's container?
2. What happens when the ResourceManager stops receiving heartbeats from a NodeManager past the expiry interval?
3. What technology coordinates leader election between Active and Standby ResourceManagers in an HA deployment?
4. What is the role of the NodeManager's ShuffleHandler auxiliary service?
5. Which property configures how much memory a NodeManager advertises as available for containers?
Was this page helpful?
You May Also Like
YARN Architecture
An overview of YARN's split-brain design that separates cluster resource management from per-application scheduling and monitoring.
Application Master
How YARN's per-application ApplicationMaster negotiates resources, drives task execution, and recovers from failure.
Scheduling in YARN
How YARN's pluggable schedulers -- FIFO, Capacity, and Fair -- decide which application gets the next available container.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingApache Flink Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
ProgrammingApache Airflow Study Notes
Programming · 30 topics
Programmingdbt (Data Build Tool) Study Notes
Programming · 30 topics
ProgrammingPowerShell Study Notes
Programming · 30 topics